Migration scripts

[djtremors]

Hi all,

I'm working on migration scripts to help people move from a normal apache,virtualmin,plesk,etc setup to ISPconfig.

I've got a script now that searchs the httpd.conf and also any included conf files for any virtualhosts and loads them into the class/array list.
It grabs all known parameters we definately use (ie ServerName, ServerAlias, DocumentRoot, etc) and some others and the rest end up as HttpdIncludes.

I've worked on plesk and virtualmin and know their structures so I could do it all (help would be great) but need help on the table and unix passwd file side of things as all this would eventually need to be linked.

anybody working on something like this? Is this somewhere we'd like to incorporate into ispc?

[till]

I think migration scripts will make the switch to ISPConfig a lot more attractive to new users.

Do plan to do the migration in one step or write a script that can be run on e.g. a plesk machine that creates a XML file with all information and this file can be loaded then into ISPConfig?

I think the approach of having universal XML file(s) containing resellers, clients websites and system users may be also useful for migrating a ISPConfig system to a new server.

We have written a migration toolkit for Cobalt RAQ servers to 42goISP, I will see if we can release this as OpenSource. It might be a good starting point.

[djtremors]

At the moment it is a single run process. The less prompting for users the better and only prompt where it's really needed.
I've got it loading up all the virtualhosts but I got problems when some servers return from the 'httpd -V' command an example list below :

.
.
-D HTTPD_ROOT="/home/httpd"
-D SUEXEC_BIN="/home/httpd/bin/suexec"
-D DEFAULT_PIDLOG="/var/logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="/var/logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/etc/httpd/conf/mime.types"
-D SERVER_CONFIG_FILE="/etc/httpd/conf/httpd.conf"

In httpd.conf file there are some lines which have :

Include conf/othervhosts.conf

Now my script checks SERVER_CONFIG_FILE and HTTPD_ROOT and in the case above. I can't find an easy way to know whats the default path for 'conf/othervhosts.conf'

We know it'll be /etc/httpd/ but programatically we can't assume it.. so i'm stuck on some servers like the above server. Others I've tested worked and an example print_r of my list is as below :

PHP Code:

*snip*[0] is default with no ServerName,etc.

config Object
(
    [httpdroot] => /home/httpd
    [httpdconf] => /etc/httpd/conf/httpd.conf
    [httpdcompiled] => Array
        (
            [APACHE_MPM_DIR] => server/mpm/prefork
            [APR_HAS_SENDFILE] =>
            [APR_HAS_MMAP] =>
            [APR_HAVE_IPV6 (IPv4-mapped addresses enabled)] =>
            [APR_USE_SYSVSEM_SERIALIZE] =>
            [APR_USE_PTHREAD_SERIALIZE] =>
            [SINGLE_LISTEN_UNSERIALIZED_ACCEPT] =>
            [APR_HAS_OTHER_CHILD] =>
            [AP_HAVE_RELIABLE_PIPED_LOGS] =>
            [HTTPD_ROOT] => /home/httpd
            [SUEXEC_BIN] => /home/httpd/bin/suexec
            [DEFAULT_PIDLOG] => /var/logs/httpd.pid
            [DEFAULT_SCOREBOARD] => logs/apache_runtime_status
            [DEFAULT_LOCKFILE] => /var/logs/accept.lock
            [DEFAULT_ERRORLOG] => logs/error_log
            [AP_TYPES_CONFIG_FILE] => /etc/httpd/conf/mime.types
            [SERVER_CONFIG_FILE] => /etc/httpd/conf/httpd.conf
        )
    [VirtualHosts] => Array
        (
            [0] => vhost Object
                (
                    [ServerName] =>
                    [ServerAlias] =>
                    [ServerAdmin] => webadmin@domain1.com
                    [DocumentRoot] => /home/httpd/html
                    [CustomLog] => /var/log/httpd/access.log Combined
                    [ErrorLog] =>
                    [AccessLog] =>
                    [HttpdIncludes] => Array
                        (
                            [0] => HostNameLookups double
                            [1] => RewriteEngine on
                            [2] => RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
                            [3] => RewriteRule .* - [F]
                        )

                )
.
.
.
            [2] => vhost Object
                (
                    [ServerName] => mail.domain3.com
                    [ServerAlias] => Array
                        (
                            [1] => mail.domain4.com.au
                            [2] => smtp.domain5.com.au
                        )

                    [ServerAdmin] => mailadmin@domain1.com
                    [DocumentRoot] => /home/httpd/squirrelmail
                    [CustomLog] => /var/log/httpd/webmail-access.log Combined
                    [ErrorLog] =>
                    [AccessLog] =>
                    [HttpdIncludes] => Array
                        (
                            [0] => HostNameLookups double
                        )

                )
.
.
.
            [23] => vhost Object
                (
                    [ServerName] => domain6.com
                    [ServerAlias] => Array
                        (
                            [1] => www.domain6.com
                        )

                    [ServerAdmin] =>
                    [DocumentRoot] =>
                    [CustomLog] =>
                    [ErrorLog] =>
                    [AccessLog] =>
                    [HttpdIncludes] => Array
                        (
                            [0] => ProxyRequests off
                            [1] => ProxyPass / http://web1.domain6.com/
                            [2] => ProxyPassReverse / http://web1.domain6.com/
                        )

                )

        )

) 


[falko]

Originally Posted by djtremors

Now my script checks SERVER_CONFIG_FILE and HTTPD_ROOT and in the case above. I can't find an easy way to know whats the default path for 'conf/othervhosts.conf'

We know it'll be /etc/httpd/ but programatically we can't assume it.. so i'm stuck on some servers like the above server. Others I've tested worked and an example print_r of my list is as below :

You can have a look at install_ispconfig/install.php because it does just the same. It detects the full path to the main Apache configuration file, and it also detects included files in the main Apache configuration file.

[till]

Here is the frontend part of the Cobalt raq migration toolkit as .zip file, the attached .zip can be installed directly (without prior unzipping) with the ISPConfig update function.

The frontend part must be installed on the server where you want to migrate to.

The package comes from 42go, I've not tested if it really works in ISPConfig
and it is currently in german, but it might help to built the frontend part in 42go for the migration script you are currently working on.

I'am still searching for the sources of the server part that is run on the cobalt raq to gather the data... I know the files must be somewhere on my harddisk

[djtremors]

The other issue i have which seems to be only some servers that are custom built are ones which use includes that have regex rules.

Include conf/websites/[.*#].conf

something like that. I dunno how i'm going to handle these. Or just detect it and alert the user to explode that list...

Is there some documentation ( and don't tell me the code itself) where there's documentation on the database structure so I know what tables added/linked, etc..?

[till]

Ok, here are the frontend files. Curious that I can not add files when I edit a post in vbulletin

[till]

Originally Posted by djtremors

Is there some documentation ( and don't tell me the code itself) where there's documentation on the database structure so I know what tables added/linked, etc..?

There is no further documentation on the database structure, but I added the files that can read a website-data dump like yours in the ISPConfig database. This can only be done in the ISPConfig interface because for example the admin has to select the customer for each website he wants to upload.

[till]

Just found the server part(s) for raq2 and raq3 that are creating the data file that is imported with the migration plugin into 42go / ISPConfig including the mnaual.

http://www.ispconfig.org/downloads/4...gration.tar.gz
http://www.ispconfig.org/downloads/4...gration.tar.gz
http://www.ispconfig.org/downloads/4.../README_EN.pdf
http://www.ispconfig.org/downloads/4.../README_DE.pdf

Maybe you can use this for your script.