PHP-Nuke and other installers [Archive] - HowtoForge Forums | HowtoForge

till

24th December 2005, 12:10

I'm developing a PHP-nuke installer in the "Tools" section. I think most installers would be very similar.
If I have a client that owns more than one web, I need a way to "know" in which site he wants to install php-nuke. From what variable can I get this info?

I dont that it will work this way, because if you know the username you cannot know in which website phpnuke should be installed. Every user can have an unlimited number of websites in ISPConfig.

You will have to let the user select in which website phpnuke shall be installed. Have a look at the backup tool, this tool shows a list of all sites the user is allowd to backup and the list to delect the phpnuke destination may be similar. Dont forget that you have also an admin and resellers and they might have to install packages for their clients too, but how to get the available sites for a user / admin / reseller this is all handled in the code for the site backups and you can copy it from there.

Also I need the user and pass to write in the database. How can I get this values? What scripts should I read?

No, dont write your own database functions. ISPCOnfig uses an database wrapper, if you use native mysql functions your extension might be incompatible to future ISPConfig releases. The database wrapper is in the class /home/admispconfig/ispconfig/lib/classes/ispconfig_mysql.lib.php (or similar name in this directory, dont have the sources here currently).

The database wrapper is already initialized if you have included the two main includes as shown in the other tools. You can call the query function e.g. by: $go_api->db->query("Here comes your SQL");

I recommend having a look at the other tools, they show you how to use the db wrapper for querys and inserts.

till

24th December 2005, 12:12

What I really need right now is to save the username of the user who executes the tool in $username. How can I accomplish this?

You can get the username from the $go_info array. Just insert print_r($go_info); in your code and you get a list of all variables available in the session.

danf.1979

24th December 2005, 19:23

OK, I love you. I'm making progress, and getting more used to the api.

danf.1979

25th December 2005, 01:31

I'm having problems creating the folders with the right permissions. The owner and group is admispconfig, I need it to be user:web[id].
This doesnt seem to work:

exec("chown $username:web1 $php_nuke_path");

Dont know why. From a console it works OK, but from the php script dont. Does this have something to do with a php feature? (i.e. security?)
How can I get to create the folder with the right permissions? I dont seem able to find the piece of code that generates de /var/www/web[ID] folder when creating a new customer. I think this code could help me. Not sure though.
I'm creating the phpnuke install folder with:

exec("mkdir $php_nuke_path");

Is this aproach the correct one?
Maybe thereis another way with the ispconfig api. Dont know for sure. I'm just starting to get used to the api.

danf.1979

25th December 2005, 02:02

Ok, progress, but still no gold.
I'm using now:

chown($php_nuke_path, $username);

But, i get this:

Warning: chown() [function.chown]: Operation not permitted in /home/admispconfig/ispconfig/web/tools/tools/test/install.php on line 66

Command was:

chown user:web1 /var/www/web1/web/bbb

Change "user" with my real user (it exists).
When I created the web directory "bbb" with mkdir, the owner and group was admispconfig. I dont seem capable o changing that, at least no from a php script.
May be I could do so with a Python script.

danf.1979

25th December 2005, 02:42

Ok, I had posted something in here, but I'll just wait for help on the last matter because I' learned already about the "->" thingy.

falko

25th December 2005, 14:42

How can I get to create the folder with the right permissions? I dont seem able to find the piece of code that generates de /var/www/web[ID] folder when creating a new customer. I think this code could help me. Not sure though.

This code is in /root/ispconfig/scripts/lib/config.lib.php. This script is run as root so you shouldn't have permission problems if you put your code in there.

danf.1979

25th December 2005, 23:17

Ok, so i put my function in /root/ispconfig/scripts/lib/config.lib.php, inside class isp_web right? but then how do I call the function from my tool?
Thank you.

danf.1979

26th December 2005, 02:48

Will I have to "include" it in my tool?

include("/root/ispconfig/scripts/lib/config.lib.php");

Or there is another way in the api to do this?

danf.1979

26th December 2005, 11:51

Ok, OK. I have been progressing a lot in the php-nuke installer.
Here you have a screenshot:
http://www.opticasqueirolo.cl/dan/phpnuke.jpg
What the script does till now, is:
Create directories and copy the all necessary files to run php-nuke in any given web. Admin, a reseller or a client can use it.
I have a problem though...
This is how my config.lib.php looks right now:

# php-nuke install function
function make_install_phpnuke($web_id, $username, $path_to_create) {

$web_id = intval($web_id);
exec("mkdir $path_to_create");
system("cp -R /var/www/installers/php-nuke/html/* $path_to_create");
system("chown -R $username:web$web_id $path_to_create");
}

class isp_web
{

var $FILE = "/root/ispconfig/scripts/lib/config.lib.php";
var $directory_mode = "0755";
var $web_doctype_id = 1013;
var $user_doctype_id = 1014;
.
.
.
etc

I have included config.lib.php in /home/admispconfig/.../tools/test2/install.php that process the form (the picture)
install.php runs fine, but I think that when included in install.php the function make_install_phpnuke() does not run with root priviledges, so the code

system("chown -R $username:web$web_id $path_to_create");

does not work. Am I right about this?

Then, I know that when you create an email user that will be the administrator, the directory permissions are changed, and you told me (falko) that only config.lib.php runned with root priviledges, so I guess the code is that changed permissions is in there. I cant find it though.
Can you help me with this?
I'm almost there... almost. And good news is that developing new installers will be very similar.
Thank you.

danf.1979

26th December 2005, 12:43

Uhm, I was looking at write.conf. I think it's structure does what I want. I found a code that teached me I had to put my function inside the class isp_web() and then make a new instance of the class in tools/test2/install.php and then call my function as a method of that class (am I right?).

Now my make_install_phpnuke() function is inside class isp_web()

class isp_web
{

var $FILE = "/root/ispconfig/scripts/lib/config.lib.php";
var $directory_mode = "0755";
var $web_doctype_id = 1013;
var $user_doctype_id = 1014;
var $domain_doctype_id = 1015;
var $dns_doctype_id = 1016;
var $a_record_doctype_id = 1018;
var $cname_record_doctype_id = 1019;
var $mx_record_doctype_id = 1020;
var $dienste_doctype_id = 1023;
var $monitor_doctype_id = 1024;
var $firewall_doctype_id = 1025;
var $slave_doctype_id = 1028;
var $datenbank_doctype_id = 1029;
var $spf_record_doctype_id = 1031;
var $vhost_conf;
var $ftp_conf;
var $apache_user;

function make_install_phpnuke($web_id, $username, $path_to_create) {

$web_id = intval($web_id);
print_r(" $web_id ");
exec("mkdir $path_to_create");
system("cp -R /var/www/installers/php-nuke/html/* $path_to_create");
print_r("chown -R $username:web$web_id $path_to_create");
system("chown -R $username:web$web_id $path_to_create");

// chown($path_to_create, $username);
// chgrp ($path_to_create, $web_id);

}

//Constructor
function isp_web() {
global $mod;
.
.
etc

My install.php has (only important code to my problem is seen)

include_once("/root/ispconfig/scripts/lib/config.lib.php");
.
.
foreach($webs as $web_id => $val) {

$query = "SELECT * from isp_nodes, sys_user where isp_nodes.doc_id = '$web_id' and isp_nodes.doctype_id = 1013 and sys_user.doc_id = isp_nodes.userid";
$get_web = $go_api->db->queryOneRecord($query);

$username = $get_web["username"];
print_r(" $username ");

$path_to_create = $httpd_root ."/web".$web_id."/web/".$install_path;
print_r(" $path_to_create ");

// NEW, call instance an method of the class (my function)
$isp_web = new isp_web;
$isp_web->make_install_phpnuke($web_id, $username, $path_to_create);
// END NEW

// if ($val==1) make_install_phpnuke($web_id, $username, $path_to_create);

}

But I execute a test a I get this:

Fatal error: Call to a member function find_includes() on a non-object in /root/ispconfig/scripts/lib/config.lib.php on line 1772

This is line 1772 (last one):

function apache_user(){
global $mod;
$httpd_conf = $mod->system->server_conf["dist_httpd_conf"];
$includes = $mod->file->find_includes($httpd_conf);
.
.
etc

Why? How do I fix this? I think this is the solution, but dont know why this error occurs...
Thanks, and sorry for the several post, but I make progress from post to post, so new problems arise, and some get fixed.

danf.1979

26th December 2005, 14:30

I commented the line
// $this->apache_user = $this->apache_user();
in function isp_web() in class isp_web, file config.lib.php, and now it gives me no error, but no permission changes either! :eek:

//Constructor
function isp_web() {
global $mod;

$this->vhost_conf = $mod->system->server_conf["server_path_httpd_conf"]."/".'vhosts'."/".'Vhosts_ispconfig.conf';
if($mod->system->server_conf["server_ftp_typ"] == "proftpd"){
$this->ftp_conf = "/etc/proftpd_ispconfig.conf";
} else {
$this->ftp_conf = "";
}
// $this->apache_user = $this->apache_user();
}

falko

26th December 2005, 14:50

You must not include /root/ispconfig/scripts/lib/config.lib.php in your install.php because then ti will not run with root privileges.

/root/ispconfig/scripts/lib/config.lib.php is called by /root/ispconfig/writeconf.php which itself is called by the /root/ispconfig/sv/ispconfig_wconf daemon which runs as root. This /root/ispconfig/sv/ispconfig_wconf daemon checks every 10 seconds if a file /home/admispconfig/ispconfig/.run exists; if it does, it calls the writeconf.php script.

So you have to modify /root/ispconfig/scripts/lib/config.lib.php and/or /root/ispconfig/scripts/writeconf.php, and whenever your install script is called, it should create /home/admispconfig/ispconfig/.run (add
touch /home/admispconfig/ispconfig/.run; to your PHP code) so that writeconf.php is called.

danf.1979

26th December 2005, 17:51

uhm, I have a very big doubt. If I cant include config.lib.php, then how can I access my function in the class isp_web?
Is there some "api" way that I'm not aware of?
Thank you for your time.

danf.1979

27th December 2005, 06:50

Do you mean I have to put my function inside config.lib.php isp_web class and then put de touch .run in that code so the deamon knows it has to run my function inside config.lib.php as root? Ok, this is what I understood from your explanation, but I dont know how to call my function from config.lib.php, since I cannot include it. I can't figure out the method to do it.
I'm really sorry if I'm being to newbie in this, but my experience is limited (I have only modified phpnuke scripts, but I know google and I dont want to give up). I think I need some guidance on this.
Thank you all.

falko

27th December 2005, 14:49

Do you mean I have to put my function inside config.lib.php isp_web class and then put de touch .run in that code so the deamon knows it has to run my function inside config.lib.php as root?
Yes.

Ok, this is what I understood from your explanation, but I dont know how to call my function from config.lib.php, since I cannot include it. I can't figure out the method to do it.

You can call your function from /root/ispconfig/scripts/writeconf.php or from any other function in the config.lib.php that you know of that it gets executed.

till

27th December 2005, 15:22

Maybe some general informations on the structure of ISPConfig will help you to get things clearer. ISPConfig consistsn of 2 parts.

1) The web interface. All files for the webinterface are in the directories /home/admispconfig/ispconfig/ and above. The scripts in the webinterface run as user admispconfig. his user has limited rights and it is not allowed to change files etc.

2) The other part are the scripts in /root/ispconfig. These where executed as root user. They where executed when the run file is set "touch /home/admispconfig/ispconfig/.run".

danf.1979

27th December 2005, 19:06

Ok! It worked! (but I have a new problem)
I did put my function on config.lib.php inside isp_web class
this is the function:

function make_install_phpnuke($web_id, $username, $path_to_create) {

$web_id = intval($web_id);
exec("mkdir $path_to_create");
system("cp -R /var/www/installers/php-nuke/html/* $path_to_create");
system("chown -R $username:web$web_id $path_to_create");
}

writeconf.php has now:

$isp_web = new isp_web;

$web_id = "8";
$username = "mundohosting";
$path_to_create = "/var/www/web8/web/phpnuke";
$isp_web->make_install_phpnuke($web_id, $username, $path_to_create);

And my install.php has the touch .run file.
Everything works as espected, BUT, as you have noticed, in writeconf.php I have defined all the functions variables...

$web_id = "8";
$username = "mundohosting";
$path_to_create = "/var/www/web8/web/phpnuke";

Because I dont know how to pass those values from install.php to writeconf.php... I cant think in a simple way to do this, but is there any? Maybe I should create a plain text file with the install data and read it from writeconf.php IF there is another touched file? like for example /home/admispconfig/.install
Is there any other way?
thank you!!

falko

27th December 2005, 20:19

I'd recommend to put those values in a database table, or even better, create a new column in the isp_isp_web table, e.g. "phpnuke" where you put either 0 or 1, and then you can read all values from the database.

danf.1979

28th December 2005, 08:30

Uhm, I'm using this query for getting the "username" that would be the "owner" of the web directory:

$go_query = "SELECT * from isp_nodes, sys_user where isp_nodes.doc_id = '$web_id' and isp_nodes.doctype_id = 1013 and sys_user.doc_id = isp_nodes.userid";

But I have realized now that this only gets the ispconfig "Login Data username" and not the "User & Email->new user->admin username" (owner of the web directory).
I had not realized this earlier because I usually named both users the same, so ispconfig login username was the same as the owner of the directory. Now I know this query does not do what I want, and that is useful only when both usernames are the same.
From the phpnuke installer *form*, only the web [ID] is passed as useful data to try to retrieve the real owner of the directory then in mysql. But I dont seem able to determine a sql query that retrieves the "user_username" in the table "isp_isp_user" with only the web [ID]. :(
Any hints?
I dont want to request the "username" to the user in the form, I want to retrieve it from mysql... Is this possible?

till

28th December 2005, 13:11

You can use this SQL query:

select isp_isp_user.user_username from isp_dep, isp_isp_user WHERE isp_dep.parent_doc_id = $web_id AND isp_dep.parent_doctype_id = 1013 AND isp_dep.child_doctype_id = 1014 AND isp_isp_user.user_admin = 1

I know its not easy to get started with ISPConfig development :)

If you need need records that depend to oather records, e.g. Web ==> User or Web ==> Co-Domain you can retrieve them by the isp_dep table. This table includes the Parent (web) doc_id's and the doc_id's of the child records like co-domains and users.

danf.1979

28th December 2005, 23:30

Thank you...
Yes, its not easy, but ispconfig is a great piece of software. That motivates me.
Well, I had to modify a bit your query. Here is the query that works in the installer:

$go_query = "SELECT isp_isp_user.user_username from isp_isp_user, isp_dep WHERE isp_dep.parent_doc_id =$web_id AND isp_dep.parent_doctype_id = 1013 AND isp_dep.child_doctype_id = 1014 AND isp_dep.child_doc_id = isp_isp_user.doc_id AND isp_isp_user.user_admin = 1 ";

Thanks for the isp_dep tip!

danf.1979

28th December 2005, 23:47

Are Mysql passwords stored some place in ispconfig tables? I would like to make an automated config.php with the name of the mysql database and the mysql password, so the user does nothing but a click to start using phpnuke...

till

28th December 2005, 23:54

No, the passwords are only stored encrypted in the mysql "mysql" system database. The best way might be if you ask the user for the password in the installer and let him select the database because every web may have multiple databases.

danf.1979

29th December 2005, 00:58

Maybe they should be stored in the database, like in table isp_server field server_db_passwort. Then we could use them for any install script we could develop for configuring automagically all configuration files necessary for the installed CMS to run. I think it would be a nice feature, because we could install everything for a given customer, even without knowing his database password.

till

29th December 2005, 01:03

If the passwords are stored in clear text in the isp_server table, it will be a massive security risk. So they are only stored in encrypted form that is irreversible.

Believe me, its better to ask the user for the password. Thats why the webFTP asks for a password too even if the user is already logged in. Never store user passwords in the database in clear text.

danf.1979

29th December 2005, 01:09

Ok, I'll think on something else then. Anyway, I see table isp_server field server_db_passwort in clear text. Is this normal?

danf.1979

29th December 2005, 08:05

Is it impossible then for an admin or reseller to install phpnuke package for a client without asking him for his password? (to configure config.php)
Suppose I get 10 or more phpnuke install request from some clients. Will I have to call all of them for asking the admin password? (suppose I'm not the admin and the actual admin does not know how to configure config.php)
Thanks.

till

29th December 2005, 10:03

In this case the admin has to set a new password. Or your installer creates automatically a new database for each installation incl. new user and pasword.

till

29th December 2005, 10:05

Ok, I'll think on something else then. Anyway, I see table isp_server field server_db_passwort in clear text. Is this normal?

Yes, because we need at least one password in clear text to access the database with the ISPConfig Interface. The special problem with userpasswords is, that most users use always the same password. So if you know their database password, you can guess e.g. their email and ftp passwords as well.

danf.1979

29th December 2005, 13:22

In this case the admin has to set a new password. Or your installer creates automatically a new database for each installation incl. new user and pasword.
Yes, I think would be a good solution. Thanks, I'll try to implement it.

danf.1979

30th December 2005, 02:11

In this case the admin has to set a new password. Or your installer creates automatically a new database for each installation incl. new user and pasword.

Could I (in some way) use the root mysql account that use ispconfig for creating new databases and *assign* those databases for a given customer?

I mean, there is already a piece of code that generates a mysql database for a customer in "optionen"/"Datenbanken Neu". I had a look to the code and found function datenbank_insert, but i also did a string search for this function in /home/admispconfig and realize that it seems that it is not called from anywhere. It exists just as a definition in ispconfig_isp_datenbank.lib.php.

How does optionen"/"Datenbanken Neu" use this function?

till

30th December 2005, 10:44

Could I (in some way) use the root mysql account that use ispconfig for creating new databases and *assign* those databases for a given customer?

Yes, thats how the "datenbank_insert function works.

I mean, there is already a piece of code that generates a mysql database for a customer in "optionen"/"Datenbanken Neu". I had a look to the code and found function datenbank_insert, but i also did a string search for this function in /home/admispconfig and realize that it seems that it is not called from anywhere. It exists just as a definition in ispconfig_isp_datenbank.lib.php.

How does optionen"/"Datenbanken Neu" use this function?

This function is called by the form for adding databases. The form definition is made with the form designer and so the function call is definied in the doctype definition and you can not find it in the code. Doctype definitions are stored in the doctype table in the database.

The framework used for ispconfig is event driven and the datenbank_insert function is called after the database add form has been saved the first time.

danf.1979

30th December 2005, 19:08

Uhm, this seems even more difficult. It is obvious for me that doctype_def is not there for direct reading. I can't understand a word of it, but I guess its a template system? I think I'll have to learn the form editor to continue ahead?

danf.1979

30th December 2005, 19:18

Uhm, ok, I had a look to the form editor form name isp_datenbank. Impressive. I'll make a new one and see what i can achieve.

till

31st December 2005, 13:58

You dont have to use the form editor. If you copy the sql queries from the datenbank_insert function that where used to create the database it will be sufficient.

danf.1979

31st December 2005, 16:59

Ok, now I ask the user for the database and his password. What I did is to insert temporarily the selected database and the user password in a table. Then writeconf.php erases the data.
Once is finished I would like to send you the script so you can tell me what you think and comment the code so I can improve it. I would really appreciate it.
I have work to do yet though. I'm searching for info on how to use the "mysqlimport" command so I can import the "nuke.sql" to populate the database.

danf.1979

31st December 2005, 17:38

Uhm, this does not work:

system("mysqlimport --user=$db_user --password=$pass_word $data_base /var/www/installers/php-nuke/sql/nuke.sql");

:confused:

falko

31st December 2005, 19:40

Have a look here: http://www.howtoforge.com/faq/6_4_en.html

danf.1979

31st December 2005, 20:21

heh, :D
Great. Thanks.

danf.1979

31st December 2005, 23:15

Does anyone have documentation or a tutorial on how to search for a given line in a file and replace it with other thing? I dont want to write an entire file to just change 3 variables...

till

1st January 2006, 12:58

If you want to perform the translations in PHP, i would open the file with fopen, read it line by line and make the replacements.

If you want to do it in a shell script or within an exec function from PHP, you can try "sed". Run "man sed" for the options on the shell. Sorry, i have no tutorial for sed, maybe you can find one with google :)

danf.1979

3rd January 2006, 20:07

Hi till, did you miss me? I did.
Well, good news... I'm almost done. BUT, I have a problem :D :
When instalation is over a message is shown to the user:

# Final message
$message = "You have now installed your wiki, but things are not over! Go with your internet browser to <a target='blank' href='$final_url'>HERE</a> and follow directions. ";
$go_api->msg($go_api->lng($message),$go_api->lng("Instalation Status"));

The problem is that the URL is something like:
https://www.mundohosting.cl:81/tools/tools/installers/www.domain.cl/installed_dir
and should be:
http://www.domain.cl/installed_dir
$final_url is http://www.domain.cl/installed_dir so I dont know what is happening... :confused:

danf.1979

3rd January 2006, 20:16

Argh, my mistake. I had to put http://$final_url, simple as that.

danf.1979

3rd January 2006, 20:24

oh, but... uhm I have another question.
How can I test if the password that the user puts in the form is the correct password for a given mysql db?

till

3rd January 2006, 22:39

You can try to do a select on the mysql user / database table like this:

$sql = "SELECT `User` FROM `mysql`.`user` WHERE `User` = '$myuser' AND `Password` = password('$mypassword')";

If the query returns the username, username and password are valid.

danf.1979

11th January 2006, 22:05

Hi again.
I have been coding a lot lately, and I'm rewriting all the installers into one big administrative interface. I'm doing Ok, but now I have a big problem.
I ask the user for the admin username and password, for example, and then I want to write those values to the *user* database, in some any table (it depends on the cms). I cannot find anything that would let me change to the user database in the mysql class.
My idea was something like this, but I doesnt work ofcourse:

$go_api->db->query("use $db_database");
$go_api->db->query("UPDATE pn_users SET pn_uname='test_user' WHERE pn_uid='2'");

And I was looking to the "query" method, and it doesnt let me change to another database...
Any help?
thanks again.

danf.1979

11th January 2006, 22:45

This worked from the command line:

mysql -h localhost -u webX_u1 -pmypass webX_db1 --execute="UPDATE pn_users SET pn_uname = 'test' WHERE pn_uid = '2'"

falko

12th January 2006, 02:36

You can have a look at the code in /root/ispconfig/scripts/lib/config.lib.php where new MySQL databases are created. Something like this should work:

$link = mysql_connect($db_server, $db_user, $db_password)
or die("Could not connect");
mysql_select_db("<db_name>")
or die("Could not select database");

$result = mysql_query("UPDATE pn_users SET pn_uname = 'test' WHERE pn_uid = '2'");

till

12th January 2006, 12:37

danf.1979

18th January 2006, 04:37

Ok, I think its time to an update on my work.
I'm doing Ok. I have developed a program called CMS Manager. It runs from its own menu in the tools section, though I would like to move it to the "ISP Manager" section.
Till now I have installers for:
<b>Portals:</b>
Etomite: http://www.etomite.org/
Joomla!: http://www.joomla.org/
PHP-Nuke: http://www.phpnuke.org/
PostNuke: http://www.postnuke.org/
WebsiteBaker: http://www.websitebaker.org/2/home/

WebsiteBaker and Etomite are great, just very simple for anyone to use with little effort.

<b>Forums</b>
phpBB
-------------------------------

I'm working on MediaWiki now.

I rewrote all my previous installers into this big cms manager (its not that big though), but from here you can now install all this programs with only some clicks and providing some required info. I dont need any more the default installers that came with the cms's tar.gz. All is done from ispconfig.

Here you have some screenshots from the Main Window (Spanish). I will rewrite it though with a different template.
http://www.opticasqueirolo.cl/dan/main.jpg
Step1, were you choose the website were to install the cms
http://www.opticasqueirolo.cl/dan/Step_1.jpg
Step2, were you enter basic info like the database name, password and directory were to install
http://www.opticasqueirolo.cl/dan/step_2.jpg

Note that every window has explanations (in spanish till now), but its very easy to go through this steps and install any software.

Some of my problems:
- When I show the user the cms_done.htm template maybe the install process has not even started yet. So the client could press the final url link that I provide and get a 404. It's a matter of time anyway. The cms does get installed.
- Till now the user has to *remember* how his database is called. I would like to do a dropdown menu for him to choose.

Ideas:
- I want to integrate this a lot more with ispconfig. Maybe 1 CMS per MySQL its OK? I think so. Maybe 2, I dont know.

till

18th January 2006, 13:34

This looks really great! :)

Some of my problems:
- When I show the user the cms_done.htm template maybe the install process has not even started yet. So the client could press the final url link that I provide and get a 404. It's a matter of time anyway. The cms does get installed.

I think there is no real solution for this problem. the best way might be to provide some info text that it may take some minutes until the CMS is installed and the link to the CMS is valid.

- Till now the user has to *remember* how his database is called. I would like to do a dropdown menu for him to choose.

This might be better, i guess many users wont remeber what they have done 10 seconds before ;)

Ideas:
- I want to integrate this a lot more with ispconfig. Maybe 1 CMS per MySQL its OK? I think so. Maybe 2, I dont know.

Better onyl one CMS per MySQL, otherwise you will get sooner or later conflicting table names. Maybe we should add limit the number of installable CMS in the Reseller limits an website settings?

danf.1979

20th January 2006, 03:36

We could add a 1 or 0 for the capabilities to install cms in the isp_isp_web_template. And from there on, 1 cms per MySQL. Maybe we should have an option too to override the 1-1 limitation too, for example if I wanted to install more than one cms (or any other ispconfig admin) for my own site (1 forum, 1 portal, 1 chat for example).
I'm going to start coding some uninstalling methods now. Do I just warn the user and erase all stuff?

danf.1979

20th January 2006, 03:39

Uhm, I think it would be unjustified to limit the reseller to only #X cms to offer. Just limit all users to 1-1 and thats all, what do you think?
I was thinking alsa about the user passwords... You said about having clear passwords to be unsecure, and I agree with that, but anyway, you have the root password in clear text. Then why dont we store the users password encoded with base_64? And in that case, clients can forget about remembering passwords.
They choose the database in the dropbox, and thats all. What about that?

falko

20th January 2006, 07:08

Uhm, I think it would be unjustified to limit the reseller to only #X cms to offer. Just limit all users to 1-1 and thats all, what do you think?
I'm not quite sure, but I can imagine that sooner or later someone will request that feature. I'm not sure though if it's enough to say "this reseller is allowed to install CMS' in his web sites, and that one isn't" instead of specifiying a number...

You said about having clear passwords to be unsecure, and I agree with that, but anyway, you have the root password in clear text.
Where do we have the root password in clear text?

till

20th January 2006, 07:16

Uhm, I think it would be unjustified to limit the reseller to only #X cms to offer. Just limit all users to 1-1 and thats all, what do you think?

It will be better if the limit can be entered as a number of cms per website.

I was thinking alsa about the user passwords... You said about having clear passwords to be unsecure, and I agree with that, but anyway, you have the root password in clear text.

The password of the root user is not used in ISPConfig at all. Maybe you mean another password that is identical with the root password in your installation.

Then why dont we store the users password encoded with base_64? And in that case, clients can forget about remembering passwords.
They choose the database in the dropbox, and thats all. What about that?

No, for security reasons we shall not store passwords in the database.

danf.1979

20th January 2006, 07:28

field server_db_passwort has the root password for the mysql root user. I see it in clear text.

till

20th January 2006, 08:40

Hmm, Ive not programmed anything waht puts the cleartext password in the DB. I will check where it is used and how we can remove it.

danf.1979

20th January 2006, 11:25

...I'm not sure though if it's enough to say "this reseller is allowed to install CMS' in his web sites, and that one isn't" instead of specifiying a number...

Ok, I agree with all you've said, but talking only about the reseller I see the following problem:
Suppose a reseller has all the tipical limitations ispconfig can give him, plus 20 CMS.
He's got 4 different plans. Now, what is the poor man going to do if the 20 CMS are over? I mean, maybe the reseller has a comertial website with his plans for everyone to see. If CMS's are over, will he be "buying" from the ispconfig admin more CMS's? or will he be modifying his plans?
I think 1 CMS per MySQL is a good choice, and reseller is CMS 1 or 0, but not number limited.
What do you think? Maybe the reseller problem has a solution, i dont know.

till

20th January 2006, 11:41

Ok, I agree with all you've said, but talking only about the reseller I see the following problem:
Suppose a reseller has all the tipical limitations ispconfig can give him, plus 20 CMS.
He's got 4 different plans. Now, what is the poor man going to do if the 20 CMS are over? I mean, maybe the reseller has a comertial website with his plans for everyone to see. If CMS's are over, will he be "buying" from the ispconfig admin more CMS's? or will he be modifying his plans?
I think 1 CMS per MySQL is a good choice, and reseller is CMS 1 or 0, but not number limited.
What do you think? Maybe the reseller problem has a solution, i dont know.

Ok, if we limit it by 1 cms = 1 mysql db, then we can limit the number of CMS by the number of databases. Thats fine too.

danf.1979

30th January 2006, 19:03

deleted... sorry. This question was my own mistake.

danf.1979

31st January 2006, 15:59

till

31st January 2006, 19:14

Hi to all.
I was just wondering about a *maybe* quota problem.

Some CMS allow uploads to certain directories, so those dirs must be world writeable. For example, CMS Made Simple allow creating directories under the upload directory, which is world writeable. Created directories are being created with owner www-data and group www-data.

My question is, do this newly created directories or uploaded files count for the quota's client? I dont think so, but I'm not really sure.

Files that belong to www-data do not coun to the web quota.

If they are not, is there any way to fix this?

Yes. But the fix is more related to your setup then to ISPCOnfig.

If you run PHP either via SuPHP or as PHP CGI with SuExec, all scripts where running under the username of web admin and not as www-data. Then all files created by CMS systems belong to the correct group and user and count to the qouta.

The drawback with SuPHP and CGI-PHP is that you loose some performance compared to mod_php.

Another possible solution might to work with PHP as fastCGI, but never tested that and it shall be not so easy to setup.

danf.1979

4th February 2006, 22:40

Uhm ok thanks. The solution is not simple because doing what you say would imply that every cms would create new folders with the right owner but the user would be unable to upload files to it because folders dont get created world writeable by the cms. Maybe a cron job would do the trick, but I'm not solving this problem right now.

I wanted to aske you something Till (or someone who knows, falko for ex). I got this code:

$get_all_db = $go_api->db->queryAllRecords("SELECT * FROM isp_isp_datenbank where doctype_id = 1029 and web_id = $web_id");

foreach($get_all_db as $db) {

$dbs .= '
<tr style="background-color: #666666;">
<td colspan="2"><span style="font-weight: bold; color: white; font-size: 13px;">
<div style="margin-left: 40px;"><input name="db_database" type="radio" value='.$db["datenbankname"].'>  '.$db["datenbankname"].'</div></span> </td>
</tr>';

$n++;
}

It generates radio buttons for the database for a given web_id. I'm not quite sure I understand the doc_id right now, I'm really being fixing and optimizing the installer code. I implemented a class for the cms_installer.php file (my own writeconf.php) but I use global statements on the methods of the class. I dont know if that would be the "correct" thing to do, but they manage to get the cms installed and that class serves to install like 10 cms rght now. Maybe you could comment on this?
Ok, back to the code. I dont really know if always a database gets installed with a 1029 doctype_id and I think that that would be the only possible failure of the mysql query right now.
I have done a very nice template for the cms installer (i think its pretty), but I know that there are other people who can do much better templates than me with css for example. Maybe some volunter to get css on this? anyone?
Ok thanks.
Oh, another question. Do i have to code some stuff to prevent sql injection in the various forms I use? I have never done this so thats why I ask. I dont know if is enough with the *general* security platform that ispconfig provides to my script.

danf.1979

4th February 2006, 23:17

Oh, something else I want to comment (not really needed right now though).
I think of ISPConfig to be the better code I've seen. Maybe it is not my code style, but I think the relationships between the scripts, the database and the daemons are really very very clever. I have little experience to say that though, and I just know little about ISPConfig code, just what I've needed to code my own script.
I have my own daemon checking for its own .signal files. With that I've managed to get a root script executed and the cms identified. I know I could do the identification with only the database, but till now the sistem works ok and the code is not very extense.
For example, cms_installer.php (my own writeconf.php) is about 275 lines, maybe 250 or 240 of code. It detects the signal file, makes necessary querys, define all necesary variables for each cms installation, and executes the install class. The install class is another script (59 lines) that executes create directories, copy, chmod, chown, rm stuff, makes the config file of the cms, gets mysql stuff done, etc.
I have done a very singular thing for the getting all this thing working. I install the cms, erase all customized field in mysql and export the sql to a file. Then even may be able of customizing this setups with new templates, pages or modified languaje files with better names and explanations for the translations. Most all of them are with the default installation though (but with my own database dump). I have most cms with default installation though, I havent have the time to make this on all cms ofcourse. With this I have been able to forgot completely about defaults installers. I maybe able in the future to choose a custom language for the cms installation, I dont know. Many things can be done with this thing I guess. I even want to make simple mysql controls for the cms manager so mysql dabase creation, cms deletions, maybe updates also, as I said, I dont know really yet what to do first. And also, I am right now optimizing code and making modifications.

How do you want to get this on ispconfig? I mean, it works in my installation, but how is the coordination for controling the possible cms to get installed going to be? Maybe I can do that with some classification in mind. I wanto to classify the cms to different classes by cms type (forum, portal, chat, etc) and with that in mind make the mysql limitations. For example, 1 MySQL permits 1 cms, and that limitation would permit only one installation for the portal, but little forum and live support center cms are too included by default. General idea would be making little cms software world wide available and bigger cms limitated. I know it can be also user configured but I will not code this in the short term.

till

4th February 2006, 23:25

Uhm ok thanks. The solution is not simple because doing what you say would imply that every cms would create new folders with the right owner but the user would be unable to upload files to it because folders dont get created world writeable by the cms. Maybe a cron job would do the trick, but I'm not solving this problem right now.

IIf you use suexec + cgiphp or suphp, the cms runs under the username of the web admin and not the apache user, so these problems dont exist and the directories must not be world writable.

if you dont use suexec + cgiphp or suphp, the direcories must be world writable.

I wanted to aske you something Till (or someone who knows, falko for ex). I got this code:

$get_all_db = $go_api->db->queryAllRecords("SELECT * FROM isp_isp_datenbank where doctype_id = 1029 and web_id = $web_id");

foreach($get_all_db as $db) {

$dbs .= '
<tr style="background-color: #666666;">
<td colspan="2"><span style="font-weight: bold; color: white; font-size: 13px;">
<div style="margin-left: 40px;"><input name="db_database" type="radio" value='.$db["datenbankname"].'>  '.$db["datenbankname"].'</div></span> </td>
</tr>';

$n++;
}

It generates radio buttons for the database for a given web_id. I'm not quite sure I understand the doc_id right now,

The doc_id is always the primary ID of a table. As the doctype_id for mysql databases is always 1029, you can optimize the query like this, but it does not harm if let it like it is now :)

SELECT * FROM isp_isp_datenbank where web_id = $web_id

I'm really being fixing and optimizing the installer code. I implemented a class for the cms_installer.php file (my own writeconf.php) but I use global statements on the methods of the class. I dont know if that would be the "correct" thing to do, but they manage to get the cms installed and that class serves to install like 10 cms rght now. Maybe you could comment on this?

Generally it is better to avoid global variables. If the codebase grows you will get lesser variable conflicts. But you dont have to change your code now, if it works.

Ok, back to the code. I dont really know if always a database gets installed with a 1029 doctype_id and I think that that would be the only possible failure of the mysql query right now.

yes, databases have always the same doctype_id 1029.

Do i have to code some stuff to prevent sql injection in the various forms I use? I have never done this so thats why I ask. I dont know if is enough with the *general* security platform that ispconfig provides to my script.

If your form is not completely generated by the form designer, you have to check all variables against SQL injection. The most secure way is by checking the values with regular expressions and escape strings correctly with the function $go_api->db->quote(".......");

webstergd

5th February 2006, 12:53

global variables, undeclared variables, and variables that are sent with post, get, cookies(basicly from the client to the server) would be the first thing an attacker will look for. It is highly recomended to never use global variables unless you really really really need to. If you do these methods you must check the variables really well.

For example, even if your variable is only used to grab an image(or just display the image name) and post it, you are running the risk of XSS attack. This was a huge problem with PostNuke, EasyNews php, webalizer, GNU Mailman, mp3 files and all sorts of programs out there. This type of attack isn't limited to images, really anythign that is posted.
(just wanted to provide an example of how dangerious user variables can be)

As till said, dont worry about changing your code if it works. I am currently going through the code, with time permitting , to help secure it.

Awesome work though. Thank you so much for doing that. If you need any help I am more then happy to help.

danf.1979

6th February 2006, 02:09

Ok thanks to you too. I'll be posting soon because my class works, but I'm using too much globals on some methods in the class. I have only programmed classes in Python and I did not use globals, but I'm rather new to php. I was looking at this moment how vars are assigned to a given class in writeconf.php. I was not aware of the sintax to do that. I think thats how I can prevent extensive use of globals. I'll give it a try right now.
I have read also something about cross site scripting and some general security topics, but I'm not on it right now. I have other things to finish yet, but I wasnt aware of those topics, so thanks. ;)

danf.1979

6th February 2006, 02:17

IIf you use suexec + cgiphp or suphp, the cms runs under the username of the web admin and not the apache user, so these problems dont exist and the directories must not be world writable.

Yes I confused me. Any drawbacks from running with those moduls? Would I have to change something in the cms installer or is it just like an apache config? Would some cms stop running? :confused: I have never used those modules. Thanks.

Note: There is another post to webstergd before this one

webstergd

6th February 2006, 05:31

possible idea...

do we have a folder or group in cvs for mod's or extra's? Might be easier for people to help with? Especially with tortoiseSVN that program is awesome...thanks for teh recomendation till.

Would it be wise to make a closed forum that only developers can read and access? We could use this for for security fixes and questions of that nature. Might be wise to restrict this to select developers who are active or "trusted."

And dude I am so looking fwd to your mod. I think it will be a huge boost for ispconfig. WAY TO GO!

danf.1979

6th February 2006, 10:54

Hi again. I just staring at the code right now and got a little scared with the function that deletes directories from a cms created installation.

function _do_delete() {
if ( is_array($this->do_delete) ) {
foreach ($this->do_delete as $value) {
system("rm -rf ".$this->path_to_create."/".$value);
}
}
}

do_delete is an array of directories to delete: $cms_install->do_delete = array("dir1/", "dir2/");
How can I be absolutely secure that I will *never* delete my entire disc?
I do define do_delete only in the script and there is no $_POST var involved, but this could not be the case in the future.
For example, could I force that all directories to eventually delete *must* be inside, for example, in /var/www/web[ID]/web/ ? (I know it can be other document root too) but just for simplicity

webstergd

6th February 2006, 11:41

you were correct in your fear. I am not sure what rights the function would be granted but it could still be a big problem.

This solution is from the top of my head with only given it a few min through so check it with Till or Falko but here is how I would make it more secure:

instead of the $value holding the directory you could use $value as a number. Then the number would triger an if statement that would then delete the coresponding directory.

i.e.
lets say $value = 2;

if( value == 1)
remove rf /var/www/web[id]/web/joomla/

else if(value == 2)
remove rf /var/www/web[id]/web/phpbb2/

else
error message

only problem with this is that web[id] would need to be properly checked to make sure it only includes proper characters ([A-Z][a-z][0-9] and I believe '_' check with Till) Have the statement die on any other values detected. Few other checks might be wise to run on web[id]. Till would be your best man to ask about the functions provided by php for this.

I still dont like web[id] in there but for simplicity sake I am not going to worry.

Later to make it easier to update you could place the list of directories in a static, readonly, config file and have the program read them and place them in a static array. still need to check the values but this should make it easier to update.

till

6th February 2006, 12:24

I think the approach from webstergd is more secure. The [ID] from web[id] is always an integer. You can check this either with an regex, e.g. "/^[0-9]{1,10}$/" or you use the fact that a valid [ID] cant be 0, so if you use somthing like $id = intval($id); will convert $id to an valid integer or will result in 0, which is harmless and can be easy filtered by if($id > 0) {....

For even more security, you might check every path right before the exec statement if it:

1) Starts with the web docroot (/home/www/ or /var/www or whatever is set in the isp_server table as root directory for the websites.
2) Does not contain 2 dots ".."
3) does contain only valid path characters. E.g. not "|<>;" and is escaped by escapeshellcmd.

Why this extra security?. The CMS installer might be extended later that it installs packeges build by external poeple / projects. If then someone builds a harmful or only lazy build package we must try to limit the possible damage as much as posiible.

Or am i too paranoid :) ?

danf.1979

6th February 2006, 23:40

Ok, I'm using this now:

function _do_delete() {

$httpd_root = "/var/www"; # This will be taken from isp_server

if(stristr($this->path_to_delete, $httpd_root."/web".$this->web_id) == TRUE AND stristr($this->path_to_delete, "..") == FALSE){
if ( is_array($this->do_delete) ) {
foreach ($this->do_delete as $value) {
print "rm -rf ".$this->path_to_delete."/".$value."<br>";
//system("rm -rf ".$this->path_to_delete."/".$value);
}
}
}
else {
echo 'Access denied.';
}
}

I dont know what do you mean by this Till: ...and is escaped by escapeshellcmd.
Thanks both.

webstergd

7th February 2006, 01:37

webstergd

7th February 2006, 05:56

possible other solution way to "secure" your statement

$value will return a url ... for example purposes we will say /var/www/web[id]/cms

you could take the variable from $value and match the string exactly with a preset string. If data isn't exact, kill the function.

if ($value == /var/www/web[id]/cms)
rm -rf /var/www/web[id]/cms
else if ($value == /var/www/web[id]/joomla)
rm -rf /var/www/web[id]/joomla
else
die

in our example the first if statement will return true and
"rm -rf /var/www/web[id]/cms" will execute.

if $value = /var/www/web[id]/cms/"insert something bad here"
the command will not execute and you are safe.

This method will be slower and less efficent then checking the given variables. However, you will not be able to execute a command you might not want to execute such as..

"rm -rf /var/www/web[id]/cms/../../../../../../../../../../../"
which is equal to "rm -rf /"

This solution is also not flexable but we could change that around later by adding how it checks. such as a searching through a static array of possible values. We could build the static array through a read only config file.

till

7th February 2006, 12:25

ehh better but still has a lot of holes.

I agree with Till on all his security points and he is a much better php programmer then I am. However, I do not feel his solution will patch all the holes in this statement.

for example:
if a users submits .../../../ he will still be able to transverse the directory. The system matches two .. not three. Called triple-dot vulnerablility.

If a hacker sends the command /var/www/.../../../../../etc/passwd you will have the password file.

Next example is that if hacker uses multiple alternate encodings for text in order to bypass the filters the filters will not flag.

/var/www/%25%25/%25%23/%25%25 ...... using URL
/var/www/%C0AE/%C0AE/%C0AF ......... using unicode

ok, I am tired so i will stop with the examples...

basicly my fear is that it is almost impossible to properly search for phrases that are not allowed. Using different encoding tricks or really just playing around you could eventully find a loophole. I am a firm believer on stating what a function can do verses what I function cannot do.

if I have time later tonight I will think of possible ways to do this that could solve your problem and make the program easier. Might not be as efficient as my original idea but should be just as secure and a hell of a lot easier to program.

SORRY TILL!!! YOU ARE STILL MY HERO!!!

I dont think that your examples will trick the filters i mentioned above:

If you have a hard coded path "/var/www/web[id]/cms" where [id] is checked / converted with the intval command or an regex and then it is passed to my filters, i dont see how this can be exploited easily. To the filters:

1) The double dot filter filters also three and mor dots when you search with stristr(...) function.

2) The filter for unallowed chars must list escape sequences too, like "%#" and others. I mentioned in my post only a few characters.

My solution was to use the solution you posted with web[id] as first "firewall" and then double check the string for malicious chars as second check before it is used in the exec statement.

The problem that i see is when we use only your type of path checking without a second "firewall", where do we get the string "/var/www/web[id]/joomla" from when the joomla package comes from an external package builder? The path string must be included in the package. If we want to have third party packages we either have to allow only "thrusted" packages where a developer from ISPConfig inspects every revision for malicoius code or we have to try to make even the installation from third party packages as secure as possible?

webstergd

7th February 2006, 20:01

Sorry Till! I miss read the post thinking it was your post verses Dan's. After reading your filters I do see your point, I appologize.

I cannot find any flaws in your web[id] filter.

I have your same fear with my filter. If it is done correctly it would be hard to allow others to expand on or allow updates. I would be time consuming to force a check for every revision of the cms's we support. However, would it be unwise to provide trusted cms packages on the website?

Complete judgement call on your part.

However, my concern was with $value. I believe holes can be punched through the filters for $value. I need to read the php documentation or ask a friend to make sure about this. But, I beleive escapseshellcmd() in php only filters single characters not double

ie
% will be kicked out but
%% will return only a single %

webstergd

7th February 2006, 20:23

I am paranoid. However, I get paid to be paranoid so I guess it is ok. :-) What you think is best Till probably will be the way to go. I trust your programming skills completely and I am sure your solution will be the best all around. Once this is up I can start hacking it and see what I get.

Just checked php's online documentation and the second post ,under the escapseshellcmd, is actually from someone who is talking about the security risk of this command. His personal recomendation was the same as mine. "actualy never accept any command from external sources only proven built-in predefined commands should be executed."

from the php documentation website:
Following characters are preceded by a backslash: #&;`|*?~<>^()[]{}$\, \x0A and \xFF. ' and " are escaped only if they are not paired.

Semi old security vulnerability on window IIS with php 4.3.6 and older: http://www.idefense.com/intelligence/vulnerabilities/display.php?id=108

till

8th February 2006, 00:37

You are right, we shall use when ever possible strict variable checking with a tightly limiting regex.

Where this is not possible, we shall consider to write a replacement for escapeshellcmd function for ISPConfig. What did you think?

webstergd

8th February 2006, 05:24

As far as rewriting escapeshellcmd goes, I think rewriting would be the best way to go. Escapeshellcmd's goal is to be a generic filter not an complete filter.

We could write one method or class that would take two variables. The first variable would be the user input variable, second variable would be what filter we would like to run. We would need to do a switch statement or if-else statements with a default method that returns a null value.

ispconfigVariableFilter(String $variable, int $checkMethod) {

if (checkMethod == 1)
//filter method 1
//check to see if $variable only contains [a-z][A-Z]
//if passes return $variable else return null
else if (checkMethod == 2)
//filter method 2
//check to see if $variable only contains [a-z][A-Z][0-9]
//if passes return $variable else return null
...

else
return null;

}

This would make it easier to modify the filter if an exploit is found. Also, helps to keep security uniform.

As far as writing filters goes I am a strong believe of stating what a variable can contain verses what it cannot. I know I say this all the time...sorry.

I want to run this by a Black Hat(hacker) programmer and see what his opinion is also. I will post back hopefully soon.

webstergd

9th February 2006, 09:24

I talked with my friend about the problem and had him read the entire thread. He is firmilar with ISP Config and has looked at some of the source code before, just not an indepth look. To add weight to his opinion, I would feel comfortable saying he could easily be one of the best "security" programmers in the US. Graduated from harvard, headhunted by google to be a security programmer(he declined), and all that goodness.

The reply from my friend goes as follows:

all non user-typed form input should with the values as numbers. These should be validated against a list of allowable values, and then used as indexes into tables that retrieve filenames and such. That way the user never has the opportunity to "fuzz" any filenames that ever get accessed directly on your system.

I think my general policy is "if it ain't in a table you created and manage carefully, it should never find it's way into a URL".

I mean, in a system ideally designed for security from the ground up you should never have to pass anything to "escapeshellcmd" because there shouldn't be any way user input would ever end up outside of your script.

In response to rewriting escapseShellCmd:
Hmmm. I like your idea of having your own input validation function that's extensible where needed, and hopefully in some reusable module.

The important thing to make sure is that the user doesn't get to control which method their input is validated against, either. So don't make it a hidden variable on the form. it should be statically set by the developer.

In response to Tills whitelist filter for web[id]:
he said that it should do the trick. In simple terms, my friend didn't find any flaws with Tills filter for web[id]. :-)