Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 2 > General

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 15th March 2006, 13:09
lorello lorello is offline
Junior Member
 
Join Date: Mar 2006
Location: Vicopisano, Pisa, Italy
Posts: 23
Thanks: 0
Thanked 0 Times in 0 Posts
Question Migrating from Ensim Webppliance to ISPConfig

Hi,
I've 100 domains and 3000 users in my old server with Ensim 3.1 and I've installed a new server with ISPConfig, now I'm thinking about migration of domains and users.

Using Ensim API I can obtain a php array with whole data about domains, users... , so I could write a script that take this array and make some query on ISPConfig database inserting records for domains & users. Is this correct? I mean, is all ispconfig data stored in mysql?

Is there some documentation about data structure of ISPConfig?

Have someone already done this ensim_to_ispc.php?
Reply With Quote
Sponsored Links
  #2  
Old 15th March 2006, 17:54
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,741 Times in 2,575 Posts
Default

You can do it with the ISPConfig remoting framework which is in beta right now and therefore not available for download. But it will be in a few weeks.
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #3  
Old 17th August 2006, 22:41
lorello lorello is offline
Junior Member
 
Join Date: Mar 2006
Location: Vicopisano, Pisa, Italy
Posts: 23
Thanks: 0
Thanked 0 Times in 0 Posts
Default I'm doing my migration :-)

Thank you Falko & Tim for Remoting Plugin!

Using "Ensim PHP Class" & "ISPConfig Remoting Plugin" I'm moving my 120 sites & 4000 users from Ensim to ISPConfig.

Doing this I've encountered some problems with ISPConfig admin interface. I've a domain with 3.500 users, when I open Users page on admin interface I get a long not paginated list of users... what about adding "pagination on Users list" in ISPConfig todolist?
Reply With Quote
  #4  
Old 18th August 2006, 10:50
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,170
Thanks: 829
Thanked 5,412 Times in 4,255 Posts
Default

Quote:
Originally Posted by lorello
Doing this I've encountered some problems with ISPConfig admin interface. I've a domain with 3.500 users, when I open Users page on admin interface I get a long not paginated list of users... what about adding "pagination on Users list" in ISPConfig todolist?
This will be really useful indeed
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #5  
Old 21st August 2006, 17:46
lorello lorello is offline
Junior Member
 
Join Date: Mar 2006
Location: Vicopisano, Pisa, Italy
Posts: 23
Thanks: 0
Thanked 0 Times in 0 Posts
Question On fat domains

Quote:
Originally Posted by till
This will be really useful indeed
In Italy we have 50% of internet connections with dialup modems at 56Kbps: users list on that domain require 4 minutes each time to load... it should be very useful :-)

Another problem with scalability: when I change a password for some user, ISPConfig make some routine from crontab that make me some doubt about scalability.

Code:
root@vandana:/# tail -f /home/admispconfig/ispconfig/ispconfig.log
21.08.2006 - 17:40:51 => INFO - USER:
web43_devgiovanna:x:11289:10043:fabio masciullo:/home/www/web43/user/web43_devgiovanna:/dev/null
21.08.2006 - 17:40:51 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web43_devgiovanna 5120 5120 0 0 -a &> /dev/null
21.08.2006 - 17:40:51 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line 57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_devgiovanna/.forward
21.08.2006 - 17:40:52 => INFO - USER:
web43_fabiogiovanna:x:11290:10043:fabio masciullo:/home/www/web43/user/web43_fabiogiovanna:/dev/null
21.08.2006 - 17:40:52 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web43_fabiogiovanna 5120 5120 0 0 -a &> /dev/null
21.08.2006 - 17:40:52 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line 57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_fabiogiovanna/.forward
21.08.2006 - 17:40:53 => INFO - USER:
web43_dfee:x:11291:10043:paola negri:/home/www/web43/user/web43_dfee:/dev/null
21.08.2006 - 17:40:53 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web43_dfee 5120 5120 0 0 -a &> /dev/null
21.08.2006 - 17:40:53 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line 57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_dfee/.forward
1 password changed, 2 seconds for 3500 users, 2 hours of hard work for ISPConfig... for one password... too much
Reply With Quote
  #6  
Old 22nd August 2006, 13:55
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,741 Times in 2,575 Posts
Default

Quote:
Originally Posted by lorello
2 hours of hard work for ISPConfig...
2 hours? Where do you see that?
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #7  
Old 22nd August 2006, 14:03
kea13 kea13 is offline
Junior Member
 
Join Date: Nov 2005
Posts: 7
Thanks: 0
Thanked 0 Times in 0 Posts
Default

Hi lorello,

since I'm stuck with an Ensim 3.1 box, too, I'd like to inquire whether you'd mind sharing this gem ?

Thanks in advance for your consideration,
Greetings,
Roman
Reply With Quote
  #8  
Old 6th September 2006, 10:25
lorello lorello is offline
Junior Member
 
Join Date: Mar 2006
Location: Vicopisano, Pisa, Italy
Posts: 23
Thanks: 0
Thanked 0 Times in 0 Posts
Default

I see the time for this operations in
/home/admispconfig/ispconfig/ispconfig.log

Code:
05.09.2006 - 16:42:06 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 254: setquota -g web
43 5120000 5120000 0 0 -a &> /dev/null
web43_manto-gas:x:13980:10043:manto-gas:/home/www/web43/user/web43_manto-gas:/dev/null
05.09.2006 - 16:42:09 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 649: cp -fr /root/is
pconfig/isp/user_standard_index.html_it /home/www/web43/user/web43_manto-gas/web/index.html
05.09.2006 - 16:42:09 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 708: setquota -u web
43_manto-gas 0 0 0 0 -a &> /dev/null
05.09.2006 - 16:42:09 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_manto-gas/.forward
05.09.2006 - 16:42:09 => INFO - maildirmake /home/www/web43/user/web43_manto-gas/Maildir &> /dev/null
, Line 109: maildirmake /home/www/web43/user/web43_manto-gas/Maildir &> /dev/null
web43_lorello:x:10195:10043:lorellone:/home/www/web43/user/web43_lorello:/dev/null
05.09.2006 - 16:42:10 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_lorello 5120 5120 0 0 -a &> /dev/null
05.09.2006 - 16:42:10 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_lorello/.forward
web43_sciarab:x:10196:10043:Michele Sciarabba:/home/www/web43/user/web43_sciarab:/dev/null
05.09.2006 - 16:42:11 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_sciarab 204800 204800 0 0 -a &> /dev/null

...

05.09.2006 - 18:34:13 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_massi.pilati 51200 51200 0 0 -a &> /dev/null
05.09.2006 - 18:34:13 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_massi.pilati/.forward
web43_manto-gas:x:13980:10043:manto-gas:/home/www/web43/user/web43_manto-gas:/dev/null
05.09.2006 - 18:34:14 => INFO - /root/ispconfig/scripts/lib/config.lib.php, Line 868: setquota -u web
43_manto-gas 0 0 0 0 -a &> /dev/null
05.09.2006 - 18:34:14 => INFO - /root/ispconfig/scripts/lib/classes/ispconfig_procmail.lib.php, Line
57: cp -f /root/ispconfig/isp/conf/forward.master /home/www/web43/user/web43_manto-gas/.forward

I've another problem with this big domain: usernames containing uppercase letters doesn't receive messages and they can't access webmail... they can't do anything :-)

In logs I can see:

Code:
05.09.2006 - 18:34:15 => WARN - /root/ispconfig/scripts/lib/config.lib.php, Line 937: WARNING: could
not setquota -u web43_Ake 0 0 0 0 -a &> /dev/null
web43_Ake:x:::::/dev/null

05.09.2006 - 18:34:15 => WARN - /root/ispconfig/scripts/lib/config.lib.php, Line 937: WARNING: could
not setquota -u web43_Micotti 0 0 0 0 -a &> /dev/null
web43_Micotti:x:::::/dev/null

05.09.2006 - 18:34:16 => WARN - /root/ispconfig/scripts/lib/config.lib.php, Line 937: WARNING: could
not setquota -u web43_Umberto.Micotti 0 0 0 0 -a &> /dev/null
web43_Umberto.Micotti:x:::::/dev/null
Reply With Quote
  #9  
Old 6th September 2006, 11:01
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,170
Thanks: 829
Thanked 5,412 Times in 4,255 Posts
Default

Uppercase letters are not supported in linux usernames, thats why you did not receive emails for them.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #10  
Old 8th September 2006, 09:30
lorello lorello is offline
Junior Member
 
Join Date: Mar 2006
Location: Vicopisano, Pisa, Italy
Posts: 23
Thanks: 0
Thanked 0 Times in 0 Posts
 
Default

Quote:
Originally Posted by kea13
Hi lorello,

since I'm stuck with an Ensim 3.1 box, too, I'd like to inquire whether you'd mind sharing this gem ?
Hi, I've not made a general purpose toolkit to make my migration, I've only used the Remoting Framework of ISPConfig to create quickly 50% of my sites, the other half has been created manually.

My old server with ensim has died when I was programming the migration and writing scripts to make it easier, so I've done many errors and if I could return on my steps I should make it better....

The steps I've followed are (should be):

- move backups of sites, database, majordomo, mailqueue, users' home, user preferences for squirrelmail from ensim box to ispconfig box
(exclude in mysql dumps the commands to create & select database, because they will change names on the ISPconfig box)

- launch many times a script that using remote framework create quickly: client, site, dns records for mail.domain.tld and mysql database (i've made it all with the same password for admin users, database etc.., we could do it better...)

- launch another script many times to restore backup of sites' files, set correct ownership & permissions.

- restore mysql database (i've done it manually, because i've made )

- recreate users on ispconfig sites: a) manually for small sites, because you don't need a default admin user as on Ensim, so you can change something during the migration; b) automatically for some bigger domains, because i've one domain with more than 3200 users...

- foreach domain, launch a script that take old mailboxes in mbox format and convert it in Maildir format (and write a log where you can read what mailboxes you have forgotten to create)
- install squirrelmail on ispconfig server and import user preferences & addressbook (i've not done it until now)

I've found two main problems after migration:

1. mixed uppercase & lowercase usernames (Some.User@some_ensim_domain.tld) that ISPConfig doesn't like.

2. ISPConfig performances running big domains: my old box was AMD 2GHz of cpu, 1Gb of ram, one ATA disk, using MailScanner (Spamassassin+clamd), my new box is a dual Xeon 3GHz with 2Gb of ram and 2 SCSI disks but if I activate spamassassin & clamd an all users, as it was on the old server, my load average jump over 200 (!!!) and the server became unusable. I think this depends on how ispconfig use spamassasin&clam (through procmail), I must work on this now to remove spam in earlier stages (on handshake, with RBLs, greylistings, etc...).

I hope this rapid notes could help you, ask on the forum if you want more info... I could have forgotten some steps.

See you,
lorello.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
cannot access ispconfig site Nu2Linux Installation/Configuration 13 3rd January 2009 15:29
Location of ISPConfig startup command cchamb2 General 3 6th April 2006 00:02
Users and websites aren't created Glorfindel Installation/Configuration 9 23rd February 2006 04:20
42goISP vs. ISPConfig nveid General 1 16th January 2006 09:07
Mirroring ISPConfig with DRBD rodriglm General 5 2nd January 2006 12:48


All times are GMT +2. The time now is 13:49.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.