HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials

HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials (http://www.howtoforge.com/forums/index.php)
-   General (http://www.howtoforge.com/forums/forumdisplay.php?f=15)
-   -   Migrating from Ensim Webppliance to ISPConfig (http://www.howtoforge.com/forums/showthread.php?t=3135)

lorello 15th March 2006 14:09

Migrating from Ensim Webppliance to ISPConfig
 
Hi,
I've 100 domains and 3000 users in my old server with Ensim 3.1 and I've installed a new server with ISPConfig, now I'm thinking about migration of domains and users.

Using Ensim API I can obtain a php array with whole data about domains, users... , so I could write a script that take this array and make some query on ISPConfig database inserting records for domains & users. Is this correct? I mean, is all ispconfig data stored in mysql?

Is there some documentation about data structure of ISPConfig?

Have someone already done this ensim_to_ispc.php? :rolleyes:

falko 15th March 2006 18:54

You can do it with the ISPConfig remoting framework which is in beta right now and therefore not available for download. But it will be in a few weeks.

lorello 17th August 2006 23:41

I'm doing my migration :-)
 
Thank you Falko & Tim for Remoting Plugin!

Using "Ensim PHP Class" & "ISPConfig Remoting Plugin" I'm moving my 120 sites & 4000 users from Ensim to ISPConfig.

Doing this I've encountered some problems with ISPConfig admin interface. I've a domain with 3.500 users, when I open Users page on admin interface I get a long not paginated list of users... what about adding "pagination on Users list" in ISPConfig todolist?

till 18th August 2006 11:50

Quote:

Originally Posted by lorello
Doing this I've encountered some problems with ISPConfig admin interface. I've a domain with 3.500 users, when I open Users page on admin interface I get a long not paginated list of users... what about adding "pagination on Users list" in ISPConfig todolist?

This will be really useful indeed :)

lorello 21st August 2006 18:46

On fat domains
 
Quote:

Originally Posted by till
This will be really useful indeed :)

In Italy we have 50% of internet connections with dialup modems at 56Kbps: users list on that domain require 4 minutes each time to load... it should be very useful :-)

Another problem with scalability: when I change a password for some user, ISPConfig make some routine from crontab that make me some doubt about scalability.

Code:

root@vandana:/# tail -f /home/admispconfig/ispconfig/ispconfig.log
21.08.2006 - 17:40:51 => INFO - USER:
web43_devgiovanna:x:11289:10043:fabio masciullo:/home/www/web43/user/web43_devgiovanna:/dev/null
21.08.2006 - 17:40:51 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web43_devgiovanna 5120 5120 0 0 -a &> /dev/null
21.08.2006 - 17:40:51 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line 57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_devgiovanna/.forward
21.08.2006 - 17:40:52 => INFO - USER:
web43_fabiogiovanna:x:11290:10043:fabio masciullo:/home/www/web43/user/web43_fabiogiovanna:/dev/null
21.08.2006 - 17:40:52 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web43_fabiogiovanna 5120 5120 0 0 -a &> /dev/null
21.08.2006 - 17:40:52 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line 57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_fabiogiovanna/.forward
21.08.2006 - 17:40:53 => INFO - USER:
web43_dfee:x:11291:10043:paola negri:/home/www/web43/user/web43_dfee:/dev/null
21.08.2006 - 17:40:53 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web43_dfee 5120 5120 0 0 -a &> /dev/null
21.08.2006 - 17:40:53 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line 57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_dfee/.forward

1 password changed, 2 seconds for 3500 users, 2 hours of hard work for ISPConfig... for one password... too much :eek:

falko 22nd August 2006 14:55

Quote:

Originally Posted by lorello
2 hours of hard work for ISPConfig...

2 hours? Where do you see that? :confused:

kea13 22nd August 2006 15:03

Hi lorello,

since I'm stuck with an Ensim 3.1 box, too, I'd like to inquire whether you'd mind sharing this gem ?

Thanks in advance for your consideration,
Greetings,
Roman

lorello 6th September 2006 11:25

I see the time for this operations in
/home/admispconfig/ispconfig/ispconfig.log

Code:

05.09.2006 - 16:42:06 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 254: setquota -g web
43 5120000 5120000 0 0 -a &> /dev/null
web43_manto-gas:x:13980:10043:manto-gas:/home/www/web43/user/web43_manto-gas:/dev/null
05.09.2006 - 16:42:09 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 649: cp -fr /root/is
pconfig/isp/user_standard_index.html_it /home/www/web43/user/web43_manto-gas/web/index.html
05.09.2006 - 16:42:09 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 708: setquota -u web
43_manto-gas 0 0 0 0 -a &> /dev/null
05.09.2006 - 16:42:09 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_manto-gas/.forward
05.09.2006 - 16:42:09 => INFO - maildirmake /home/www/web43/user/web43_manto-gas/Maildir &> /dev/null
, Line 109: maildirmake /home/www/web43/user/web43_manto-gas/Maildir &> /dev/null
web43_lorello:x:10195:10043:lorellone:/home/www/web43/user/web43_lorello:/dev/null
05.09.2006 - 16:42:10 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_lorello 5120 5120 0 0 -a &> /dev/null
05.09.2006 - 16:42:10 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_lorello/.forward
web43_sciarab:x:10196:10043:Michele Sciarabba:/home/www/web43/user/web43_sciarab:/dev/null
05.09.2006 - 16:42:11 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_sciarab 204800 204800 0 0 -a &> /dev/null

...

05.09.2006 - 18:34:13 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_massi.pilati 51200 51200 0 0 -a &> /dev/null
05.09.2006 - 18:34:13 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_massi.pilati/.forward
web43_manto-gas:x:13980:10043:manto-gas:/home/www/web43/user/web43_manto-gas:/dev/null
05.09.2006 - 18:34:14 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_manto-gas 0 0 0 0 -a &> /dev/null
05.09.2006 - 18:34:14 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_manto-gas/.forward


I've another problem with this big domain: usernames containing uppercase letters doesn't receive messages and they can't access webmail... they can't do anything :-)

In logs I can see:

Code:

05.09.2006 - 18:34:15 => WARN - /root/ispconfig/scripts/lib/config.lib.php, Line 937: WARNING: could
not setquota -u web43_Ake 0 0 0 0 -a &> /dev/null
web43_Ake:x:::::/dev/null

05.09.2006 - 18:34:15 => WARN - /root/ispconfig/scripts/lib/config.lib.php, Line 937: WARNING: could
not setquota -u web43_Micotti 0 0 0 0 -a &> /dev/null
web43_Micotti:x:::::/dev/null

05.09.2006 - 18:34:16 => WARN - /root/ispconfig/scripts/lib/config.lib.php, Line 937: WARNING: could
not setquota -u web43_Umberto.Micotti 0 0 0 0 -a &> /dev/null
web43_Umberto.Micotti:x:::::/dev/null


till 6th September 2006 12:01

Uppercase letters are not supported in linux usernames, thats why you did not receive emails for them.

lorello 8th September 2006 10:30

Quote:

Originally Posted by kea13
Hi lorello,

since I'm stuck with an Ensim 3.1 box, too, I'd like to inquire whether you'd mind sharing this gem ?

Hi, I've not made a general purpose toolkit to make my migration, I've only used the Remoting Framework of ISPConfig to create quickly 50% of my sites, the other half has been created manually.

My old server with ensim has died when I was programming the migration and writing scripts to make it easier, so I've done many errors and if I could return on my steps I should make it better....

The steps I've followed are (should be):

- move backups of sites, database, majordomo, mailqueue, users' home, user preferences for squirrelmail from ensim box to ispconfig box
(exclude in mysql dumps the commands to create & select database, because they will change names on the ISPconfig box)

- launch many times a script that using remote framework create quickly: client, site, dns records for mail.domain.tld and mysql database (i've made it all with the same password for admin users, database etc.., we could do it better...)

- launch another script many times to restore backup of sites' files, set correct ownership & permissions.

- restore mysql database (i've done it manually, because i've made )

- recreate users on ispconfig sites: a) manually for small sites, because you don't need a default admin user as on Ensim, so you can change something during the migration; b) automatically for some bigger domains, because i've one domain with more than 3200 users...

- foreach domain, launch a script that take old mailboxes in mbox format and convert it in Maildir format (and write a log where you can read what mailboxes you have forgotten to create)
- install squirrelmail on ispconfig server and import user preferences & addressbook (i've not done it until now)

I've found two main problems after migration:

1. mixed uppercase & lowercase usernames (Some.User@some_ensim_domain.tld) that ISPConfig doesn't like.

2. ISPConfig performances running big domains: my old box was AMD 2GHz of cpu, 1Gb of ram, one ATA disk, using MailScanner (Spamassassin+clamd), my new box is a dual Xeon 3GHz with 2Gb of ram and 2 SCSI disks but if I activate spamassassin & clamd an all users, as it was on the old server, my load average jump over 200 (!!!) and the server became unusable. I think this depends on how ispconfig use spamassasin&clam (through procmail), I must work on this now to remove spam in earlier stages (on handshake, with RBLs, greylistings, etc...).

I hope this rapid notes could help you, ask on the forum if you want more info... I could have forgotten some steps.

See you,
lorello.


All times are GMT +2. The time now is 13:09.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.