#1  
Old 24th July 2006, 07:57
djtremors djtremors is offline
Senior Member
 
Join Date: Apr 2006
Location: Sydney
Posts: 278
Thanks: 0
Thanked 12 Times in 10 Posts
Default Migration scripts

Hi all,

I'm working on migration scripts to help people move from a normal apache,virtualmin,plesk,etc setup to ISPconfig.

I've got a script now that searchs the httpd.conf and also any included conf files for any virtualhosts and loads them into the class/array list.
It grabs all known parameters we definately use (ie ServerName, ServerAlias, DocumentRoot, etc) and some others and the rest end up as HttpdIncludes.

I've worked on plesk and virtualmin and know their structures so I could do it all (help would be great) but need help on the table and unix passwd file side of things as all this would eventually need to be linked.

anybody working on something like this? Is this somewhere we'd like to incorporate into ispc?
Reply With Quote
Sponsored Links
  #2  
Old 24th July 2006, 10:49
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,586
Thanks: 792
Thanked 4,983 Times in 3,903 Posts
Default

I think migration scripts will make the switch to ISPConfig a lot more attractive to new users.

Do plan to do the migration in one step or write a script that can be run on e.g. a plesk machine that creates a XML file with all information and this file can be loaded then into ISPConfig?

I think the approach of having universal XML file(s) containing resellers, clients websites and system users may be also useful for migrating a ISPConfig system to a new server.

We have written a migration toolkit for Cobalt RAQ servers to 42goISP, I will see if we can release this as OpenSource. It might be a good starting point.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.

Last edited by till; 24th July 2006 at 10:52.
Reply With Quote
  #3  
Old 25th July 2006, 13:13
djtremors djtremors is offline
Senior Member
 
Join Date: Apr 2006
Location: Sydney
Posts: 278
Thanks: 0
Thanked 12 Times in 10 Posts
Default

At the moment it is a single run process. The less prompting for users the better and only prompt where it's really needed.
I've got it loading up all the virtualhosts but I got problems when some servers return from the 'httpd -V' command an example list below :

.
.
-D HTTPD_ROOT="/home/httpd"
-D SUEXEC_BIN="/home/httpd/bin/suexec"
-D DEFAULT_PIDLOG="/var/logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="/var/logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/etc/httpd/conf/mime.types"
-D SERVER_CONFIG_FILE="/etc/httpd/conf/httpd.conf"

In httpd.conf file there are some lines which have :

Include conf/othervhosts.conf

Now my script checks SERVER_CONFIG_FILE and HTTPD_ROOT and in the case above. I can't find an easy way to know whats the default path for 'conf/othervhosts.conf'

We know it'll be /etc/httpd/ but programatically we can't assume it.. so i'm stuck on some servers like the above server. Others I've tested worked and an example print_r of my list is as below :

PHP Code:
*snip*[0is default with no ServerName,etc.

config Object
(
    [
httpdroot] => /home/httpd
    
[httpdconf] => /etc/httpd/conf/httpd.conf
    
[httpdcompiled] => Array
        (
            [
APACHE_MPM_DIR] => server/mpm/prefork
            
[APR_HAS_SENDFILE] =>
            [
APR_HAS_MMAP] =>
            [
APR_HAVE_IPV6 (IPv4-mapped addresses enabled)] =>
            [
APR_USE_SYSVSEM_SERIALIZE] =>
            [
APR_USE_PTHREAD_SERIALIZE] =>
            [
SINGLE_LISTEN_UNSERIALIZED_ACCEPT] =>
            [
APR_HAS_OTHER_CHILD] =>
            [
AP_HAVE_RELIABLE_PIPED_LOGS] =>
            [
HTTPD_ROOT] => /home/httpd
            
[SUEXEC_BIN] => /home/httpd/bin/suexec
            
[DEFAULT_PIDLOG] => /var/logs/httpd.pid
            
[DEFAULT_SCOREBOARD] => logs/apache_runtime_status
            
[DEFAULT_LOCKFILE] => /var/logs/accept.lock
            
[DEFAULT_ERRORLOG] => logs/error_log
            
[AP_TYPES_CONFIG_FILE] => /etc/httpd/conf/mime.types
            
[SERVER_CONFIG_FILE] => /etc/httpd/conf/httpd.conf
        
)
    [
VirtualHosts] => Array
        (
            [
0] => vhost Object
                
(
                    [
ServerName] =>
                    [
ServerAlias] =>
                    [
ServerAdmin] => webadmin@domain1.com
                    
[DocumentRoot] => /home/httpd/html
                    
[CustomLog] => /var/log/httpd/access.log Combined
                    
[ErrorLog] =>
                    [
AccessLog] =>
                    [
HttpdIncludes] => Array
                        (
                            [
0] => HostNameLookups double
                            
[1] => RewriteEngine on
                            
[2] => RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
                            [
3] => RewriteRule .* - [F]
                        )

                )
.
.
.
            [
2] => vhost Object
                
(
                    [
ServerName] => mail.domain3.com
                    
[ServerAlias] => Array
                        (
                            [
1] => mail.domain4.com.au
                            
[2] => smtp.domain5.com.au
                        
)

                    [
ServerAdmin] => mailadmin@domain1.com
                    
[DocumentRoot] => /home/httpd/squirrelmail
                    
[CustomLog] => /var/log/httpd/webmail-access.log Combined
                    
[ErrorLog] =>
                    [
AccessLog] =>
                    [
HttpdIncludes] => Array
                        (
                            [
0] => HostNameLookups double
                        
)

                )
.
.
.
            [
23] => vhost Object
                
(
                    [
ServerName] => domain6.com
                    
[ServerAlias] => Array
                        (
                            [
1] => www.domain6.com
                        
)

                    [
ServerAdmin] =>
                    [
DocumentRoot] =>
                    [
CustomLog] =>
                    [
ErrorLog] =>
                    [
AccessLog] =>
                    [
HttpdIncludes] => Array
                        (
                            [
0] => ProxyRequests off
                            
[1] => ProxyPass http://web1.domain6.com/
                            
[2] => ProxyPassReverse http://web1.domain6.com/
                        
)

                )

        )


Reply With Quote
  #4  
Old 25th July 2006, 13:29
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 41,711
Thanks: 1,899
Thanked 2,702 Times in 2,545 Posts
Default

Quote:
Originally Posted by djtremors
Now my script checks SERVER_CONFIG_FILE and HTTPD_ROOT and in the case above. I can't find an easy way to know whats the default path for 'conf/othervhosts.conf'

We know it'll be /etc/httpd/ but programatically we can't assume it.. so i'm stuck on some servers like the above server. Others I've tested worked and an example print_r of my list is as below :
You can have a look at install_ispconfig/install.php because it does just the same. It detects the full path to the main Apache configuration file, and it also detects included files in the main Apache configuration file.
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #5  
Old 25th July 2006, 13:36
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,586
Thanks: 792
Thanked 4,983 Times in 3,903 Posts
Default

Here is the frontend part of the Cobalt raq migration toolkit as .zip file, the attached .zip can be installed directly (without prior unzipping) with the ISPConfig update function.

The frontend part must be installed on the server where you want to migrate to.

The package comes from 42go, I've not tested if it really works in ISPConfig
and it is currently in german, but it might help to built the frontend part in 42go for the migration script you are currently working on.

I'am still searching for the sources of the server part that is run on the cobalt raq to gather the data... I know the files must be somewhere on my harddisk
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #6  
Old 25th July 2006, 13:37
djtremors djtremors is offline
Senior Member
 
Join Date: Apr 2006
Location: Sydney
Posts: 278
Thanks: 0
Thanked 12 Times in 10 Posts
Default

The other issue i have which seems to be only some servers that are custom built are ones which use includes that have regex rules.

Include conf/websites/[.*#].conf

something like that. I dunno how i'm going to handle these. Or just detect it and alert the user to explode that list...

Is there some documentation ( and don't tell me the code itself) where there's documentation on the database structure so I know what tables added/linked, etc..?
Reply With Quote
  #7  
Old 25th July 2006, 13:37
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,586
Thanks: 792
Thanked 4,983 Times in 3,903 Posts
Default

Ok, here are the frontend files. Curious that I can not add files when I edit a post in vbulletin
Attached Files
File Type: zip migration.zip (7.7 KB, 396 views)
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #8  
Old 25th July 2006, 13:41
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,586
Thanks: 792
Thanked 4,983 Times in 3,903 Posts
Default

Quote:
Originally Posted by djtremors
Is there some documentation ( and don't tell me the code itself) where there's documentation on the database structure so I know what tables added/linked, etc..?
There is no further documentation on the database structure, but I added the files that can read a website-data dump like yours in the ISPConfig database. This can only be done in the ISPConfig interface because for example the admin has to select the customer for each website he wants to upload.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #9  
Old 25th July 2006, 14:00
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,586
Thanks: 792
Thanked 4,983 Times in 3,903 Posts
 
Default

Just found the server part(s) for raq2 and raq3 that are creating the data file that is imported with the migration plugin into 42go / ISPConfig including the mnaual.

http://www.ispconfig.org/downloads/4...gration.tar.gz
http://www.ispconfig.org/downloads/4...gration.tar.gz
http://www.ispconfig.org/downloads/4.../README_EN.pdf
http://www.ispconfig.org/downloads/4.../README_DE.pdf

Maybe you can use this for your script.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Simple Scripts for first time install. pohl Installation/Configuration 5 21st May 2006 20:59
PHP scripts don't work, please help. CombatGod Installation/Configuration 9 11th May 2006 17:37
PERL scripts in Mozilla Browser Deepthi Server Operation 3 20th December 2005 14:01
Migration mphayesuk Feature Requests 0 16th November 2005 21:59
Server Migration adamluz Feature Requests 0 15th August 2005 20:54


All times are GMT +2. The time now is 06:58.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.