CLI and web_domain "save" action

[osmoza]

Hello,

I've got several CLI scripts that inserts data into ISPConfig's database

Something like that:

$ user_add client_name username password other_stuff && web_domain_add client_name domain.name.tld

Now, I'd like to force ISPConfigs to generate all the needed system (web_domain related to) files and inserts (vhosts, passwd, shadow etc.) for database inserts made with this scripts ("call the save action" in the web_domain form, but via CLI).

Is it possible? Is there a function/method/script that does the trick? If so - where can I find it?

Cheers,
Osmoza

[till]

Not sure how your cli scripts work as ispconfig does not provide any cli scripts. If they use the remoting framework to connect to ispconfig, then there is no further action needed. if they try to insert sql records manually, then this is the source of your problem. You should rewrite them to use the remoting API as the APi ensures that the records were inserted correctly incl. correct permission settings and thet the config files are written based on these records.

[osmoza]

Hello Till,

The scripts inserts sql manually - I know that's cousing my problem.
The question is: how can I get to methods/functions/scripts (as you mentioned - the last one doesn't exist) that parses db records and saves them in the system.

I've found processDatalog() method in modules class and server.sh script - I guess, I'm in need of saving sys_datalog and running server.sh to do "if( changes found ) { generate stuff }"....

Remote API? I'll give it a try BUT I've made some simple changes in ISPConfig's source (changes in web_domain db schema - added field, vhost template changed.....) and I'm affraid that it's not gonna work without messing with API's source too (maybe I'm wrong).

Thanks,
Osmoza

[till]

Looks as you mix up here several things. This thread here is about ispconfig 2 (its in the ispconfig 2 forum) and ispconfig 2 does not has a function processDatalog. ISPConfig 3 has this function, but thats a completely different software that works differently.

So which ispconfig version do you use?

I've made some simple changes in ISPConfig's source (changes in web_domain db schema - added field, vhost template changed.....) and I'm affraid that it's not gonna work without messing with API's source too (maybe I'm wrong).

I hope you are aware that you are not able to install any ptches or updates for ispconfig anymore as the updater will remove your complete databse on the next update if your databae scheme is different to the one from ispconfig. So adding any fields is a bad idea if you want to benefit from new features in future ispconfig versions.

[osmoza]

Oh sh**!
You are right - I'm using ISPConfig3 - didn't notice "ISPConfig 2" in breadcrumbs...
Could you please move thread to proper room/category (whatever it's called)?

Updates and stuff - I know, but don't need them as much, as I need extra functionality I get with these changes (long story).

Thanks,
O

[till]

Take a look at the file /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php

Hou have to use the dalaogInsert, datalogUpdate and datalogDelete functions from there to do any kind of data manipulation in the ispconfig database. Everything that you uppate with sql queries without using these functions will be ignored by the server script and not be used to write any config files.

[osmoza]

OK, done
Thanks for help Till.

The three methods you privided sets sys_datalog and that makes the whole magic!

As far as I see - all I had to do to write simple Ispconfig API is calling datalogSave() each time I change something with database (as you said). So...I've changed my Database::save() method to call this method and....that's it - everything works fine

I had to redefine some things in your db class though (the "global $app.." in methods - what's that for? some backward compatibility stuff?) - the code's included (with two or three calls to Database::query and some $modelObject->save() methods...- original versions in comments, so should work with 2-3 little changes).

I also included little script that creates client (for particular reseller), and sets website for that client - example of how and what for I'm using it for.

If you're not interested in the code - that's all I had to say - no futher reading needed
Thanks again
O.

THE CODE:

db_mysql.inc.php redefined as helper class:

PHP Code:

/* ISPCONFIG's EULA */
class Ispconfig
{
    /** Returns the last mySQL insert_id() */
    static public function insertID()
    {
        return mysql_insert_id();
    }


    /** Escapes quotes in variable. mysql_real_escape_string() */
    static public function quote($formfield)
    {
        return mysql_real_escape_string($formfield);
    }

    /**
     *  Function to fill the datalog with a full differential record.
     * 
     * @param <type> $db_table  -   table name
     * @param <type> $action    -   INSERT/UPDATE/DELETE...
     * @param String $primary_field -   primary field's name (ex. client_id)
     * @param <type> $primary_id    -   primary fielr's value (ex. 21)
     * @param array $record_old    -   array representing record (ex. array('client_id'=>'21'))
     * @param array $record_new    -   array representing record (ex. array('client_id'=>'21'))
     * @return <Boolean>
     */
    static public function datalogSave($db_table, $action, $primary_field, $primary_id, $record_old, $record_new)
    {
        // Insert backticks only for incomplete table names.
        if(stristr($db_table,'.'))
        {
            $escape = '';
        } else
        {
            $escape = '`';
        }

        $diffrec_full = array();
        $diff_num = 0;

        if(is_array($record_old) && count($record_old) > 0)
        {
            foreach($record_old as $key => $val)
            {
                if(!isset($record_new[$key]) || $record_new[$key] != $val)
                {
                // Record has changed
                    $diffrec_full['old'][$key] = $val;
                    $diffrec_full['new'][$key] = $record_new[$key];
                    $diff_num++;
                } else
                {
                    $diffrec_full['old'][$key] = $val;
                    $diffrec_full['new'][$key] = $val;
                }
            }
        } elseif(is_array($record_new))
        {
            foreach($record_new as $key => $val)
            {
                if(isset($record_new[$key]) && @$record_old[$key] != $val)
                {
                // Record has changed
                    $diffrec_full['new'][$key] = $val;
                    $diffrec_full['old'][$key] = @$record_old[$key];
                    $diff_num++;
                } else
                {
                    $diffrec_full['new'][$key] = $val;
                    $diffrec_full['old'][$key] = $val;
                }
            }
        }

        // Insert the server_id, if the record has a server_id
        $server_id = (isset($record_old["server_id"]) && $record_old["server_id"] > 0)?$record_old["server_id"]:SERVER_ID;
        if(isset($record_new["server_id"])) $server_id = $record_new["server_id"];


        if($diff_num > 0)
        {
            $diffstr = self::quote(serialize($diffrec_full));
            $reseller = Client::findByPk(CLIENTS_PARENT_CLIENT_ID); //get defined reseller instead of currently logged in user
            $username = self::quote($reseller['username']);
//            $username = self::quote($_SESSION["s"]["user"]["username"]);  //get currently loged in username
            $dbidx = $primary_field.":".$primary_id;

            if($action == 'INSERT') $action = 'i';
            if($action == 'UPDATE') $action = 'u';
            if($action == 'DELETE') $action = 'd';
            $sql = "INSERT INTO sys_datalog (dbtable,dbidx,server_id,action,tstamp,user,data) VALUES ('".$db_table."','$dbidx','$server_id','$action','".time()."','$username','$diffstr')";
            Database::query($sql);  //process query - similar to $app->db->query($sql);
        }
        return true;
    }

} 


Changed my Database::save() implementation so that it works with Ispconfig's sys_datalog:

PHP Code:

/* ...method definition.... */
//ISPCONFIG DALTA_LOG FIX
        if($object->isNew())
        {
            $old_rec = array(); //new object, so old_rec is empty
            $query_result = self::query($sql);
            $index_value = Ispconfig::insertID();
            $object->fields[$object->getPrimaryKey()]['value'] = $index_value;  //XXX WANDER IF THIS WORKS :)
            $new_recs = self::query('SELECT * FROM '.$object->getTable().' WHERE '.$object->getPrimaryKey().' = \''.$index_value.'\';');
            $new_rec = $new_recs[0];
            if(Ispconfig::datalogSave($object->getTable(), 'INSERT', $object->getPrimaryKey(), $index_value, $old_rec, $new_rec) == true)
            {
                return $query_result;
            }
            else
            {
                return false;
            }
        }
        else
        {
            $old_recs = self::query('SELECT * FROM '.$object->getTable().' WHERE '.$object->getPrimaryKey().'='.$object->fields[$object->getPrimaryKey()]['value'].';');
            $old_rec = $old_recs[0];
            $query_result = self::query($sql);
            $new_recs = self::query('SELECT * FROM '.$object->getTable().' WHERE '.$object->getPrimaryKey().'='.$object->fields[$object->getPrimaryKey()]['value'].';');
            $new_rec = $new_recs[0];
            if(Ispconfig::datalogSave($object->getTable(), 'UPDATE', $object->getPrimaryKey(), $object->fields[$object->getPrimaryKey()]['value'], $old_rec, $new_rec) == true)
            {
                return $query_result;
            }
            else
            {
                return false;
            }
        } 


And finally - simple use case (adds Client, creates required stuff like sys_user, sys_group, adds web_domain):

PHP Code:

<?php
include('/home/user/system/config/Constants.php');
include(AUTOLOADER);
include(DBINITER);

$paramNumber = 4;
$errors = array();
$errors['no_required_params'] = <<<EOF
Provide this parameters:
email   -   users email - used for client's name nad contact's name.
username    - ispconfig's username
password    - username's password
domain  -   domain as xxx.yyyyy.tld

EOF;
if((count($argv) -1) != $paramNumber)
{
    echo $errors['no_required_params'];
    exit;
}

//Filter, escape, validate provided parameters...then set them
$email = $argv[1];
$username = $argv[2];
$password = $argv[3];
$domain = $argv[4];

$company_name = $email;
$contact_name = $email;

//ClientsParent 
$clientsParent = Client::findByPk(CLIENTS_PARENT_CLIENT_ID,true);    //client's parent object
$clientParentSysGroup = SysGroup::findOneBy('client_id',CLIENTS_PARENT_CLIENT_ID,true);  //client's parent's sysgroup
$clientParentSysUser = SysUser::findOneBy('client_id',CLIENTS_PARENT_CLIENT_ID,true);    //clients's pranent's sysuser

//new client
$client = new Client();
$client->fields['company_name']['value']   =   $company_name;
$client->fields['contact_name']['value'] =   $contact_name;
$client->fields['username']['value']   =   $username;
$client->fields['password']['value'] =   crypt($password);
$client->fields['parent_client_id']['value'] =   CLIENTS_PARENT_CLIENT_ID;

$status = ($client->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'CLIENT_OBJECT_SAVE: '.$status;
    exit;
}

//Creating sys_group for this client
$sysgroup = new SysGroup();
$sysgroup->fields['name']['value']  =   $client->fields['username']['value'];
$sysgroup->fields['client_id']['value'] = $client->fields['client_id']['value'];

$status = ($sysgroup->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'SYS_GROUP_OBJECT_SAVE: '.$status;
    exit;
}


//Creating sys_user with privided sysgroup's ID and client's ID
$sysuser = new SysUser();
$sysuser->fields['username']['value']    =   $username;
$sysuser->fields['passwort']['value']    =   md5($password);
$sysuser->fields['client_id']['value']   =   $client->fields['client_id']['value'];
$sysuser->fields['groups']['value'] = $sysgroup->fields['groupid']['value'];
$sysuser->fields['default_group']['value'] = $sysgroup->fields['groupid']['value'];

$status = ($sysuser->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'SYS_USER_OBJECT_SAVE: '.$status;
    exit;
}

//Updates client with sys_groupid and sys_userid fields with provided data
$client->fields['sys_userid']['value']   =   $clientParentSysUser->fields['userid']['value'];
$client->fields['sys_groupid']['value']  =   $clientParentSysGroup->fields['groupid']['value'];
$status = ($client->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'SYS_USER_OBJECT_SAVE: '.$status;
    exit;
}

//Updates Clients parent sys_user
$clientParentSysUser->fields['groups']['value']   .=  ','.$sysgroup->fields['groupid']['value'];
$status = ($clientParentSysUser->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'SYS_USER_PARENT_OBJECT_SAVE: '.$status;
    exit;
}

//WEB DOMAIN
$clientSysGroup = SysGroup::findOneBy('client_id',$client->fields['client_id']['value']);

$webdomain = new WebDomain();
$webdomain->fields['sys_userid']['value'] = $clientParentSysUser->fields['userid']['value'];
$webdomain->fields['sys_groupid']['value'] = $clientSysGroup['groupid'];
$webdomain->fields['domain']['value'] = $domain;
$webdomain->fields['system_group']['value'] = 'client'.$client->fields['client_id']['value'];
$status = ($webdomain->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'WEB_DOMAIN_ADD: '.$status;
    exit;
}

$webdomain->fields['system_user']['value']  =   'web'.$webdomain->fields['domain_id']['value'];
$system_group = $webdomain->fields['system_group']['value'];
$system_user = $webdomain->fields['system_user']['value'];
$webdomain->fields['document_root']['value']  =   '/var/www/clients/'.$system_group.'/'.$system_user;
$status = ($webdomain->save() === true) ? 'SUCCESS' :   'FAILURE';
if($status == 'FAILURE')
{
    echo 'WEB_DOMAIN_ADD_UPDATE: '.$status;
    exit;
}
echo 'SUCCESS';
?>

[till]

The $app object is the base object of the ispconfig system. It is used for everything, e.g. dynamic loading of calsses, the language system and also for accessing class functions.

[osmoza]

OK, but why not using something like Application::getApplication() - method that returns application object?

For (backward) comatibility you could use it as:

PHP Code:

public function foo()
{
    $app = Application::getApplication();
   $app->db->query(....);
    .....
} 


Same thing with "global $config" - global in methods seams less OOP for me. One have no idea what global $app/$config is and where to look for it. With Application::getApplication() - you can "RMB > Navigate to source" and get all the information needed.

You could also copy-paste Application class with DB class only and have all the functionality both of them provide. With global $app - one have no idea what to copy and how to call it (how to instantiate $app, where to do it, and so on..).

Oh, and FYI: I'm not dissing your work! Ispconfig saved me houndreds of hours of developing - I love it!

I'm asking about design decission - the above is in my opinion beter approach, but I might be wrong.

Cheers,
O.

[till]

The $app object is initialized when the framework is loaded, so no need to initialize it. Using it globally is just a line of code of nearly the same length then loading it in every function. All classes are availble as $app->classname->function(), so code is very easy to navigate.

One have no idea what global $app/$config is

Everyone developing in ispconfig and looked at the code before knows it as its the basic principle of this software. ISPConfig requires a coding style and the decision to use a global $app and $config object is one of the coding standards of this project.

The ispconfig classes are not made and its not intended to use them in other external applications. They are just written to work as part of the ispconfig framework.