Migrate from Courier/Lenny to Dovecot/Wheezy

Discussion in 'Installation/Configuration' started by James A, Jan 7, 2014.

  1. James A

    James A Member HowtoForge Supporter

    Hi,

    I'm looking to update our mail servers which are now long in the tooth.

    As part of the upgrade I'd like to move from using courier to dovecot as the overall volume of mail being stored per user is causing courier to grind badly in webmail. Current servers are running ISPconfig 3.0.3.2

    My question is how would you do the change on live servers?

    1) Create new Debian Wheezy server with Dovecot running on it and import the mail somehow. How would you keep user account details, mail UID etc, some of the users have very slow connections and large mailboxes so I can't have them redownload the mail again.

    2) Create a new Debian Wheezy server with Courier and simply copy the var/vmail folders and ISPconfig database to the new server, then migrate the new server from Courier to Dovecot. Has anyone got a decent description of how to migrate from courier to dovecot with ISPconfig.

    3) Any other thoughts

    I have tried working through option 2 but have had issues with the dovecot migration where I'm getting stuck having updated the folders using courier-dovecot-migrate.pl and then installing dovecot (apt-get install dovecot-imapd dovecot-pop3d). I think the issue is simply in ISPconfig where I tell it to use Dovecot but am then a little confused as to what to set for mail filter syntax and how to get that working if it needs to be sieve. Anyway my misconfiguration results in an authentication error when trying to login using roundcube.

    Any help or advice is greatly appreciated.
     
  2. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    I would first update to latest ISPConfig I think.
    Then set up new wheezy server with dovecot instead of courier and install latest ISPConfig there, too.
    Then migrate ISPConfig data and mail data and use the dovecot migration script (use the correct version! there is one for dovecot and one for dovecot2).
     
  3. James A

    James A Member HowtoForge Supporter

    Thanks for the reply Croydon.

    would you see any potential issues in the ISPconfig upgrade, it's running on a live server hence a little aprehensive about the upgrade?

    With regard to the dovecot versions I'm assuming the The Perfect Server – Debian Wheezy (Apache2, BIND, Dovecot, ISPConfig 3) is using dovecot not dovecot 2.

    Is there anything else I need to do to the mail data, ie would you simply add all folders to the var/vmail directory and run the update from here . Where I'm not so sure is with the folder structure, I remember reading that dovecot was based on a domain/user/maildir where courier uses domain/user only, would I have to correct this for each folder?

    Final thing, is there a good reason to go for dovecot 2?:eek:
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Normally there arent any problems and it is highly recommended to update regularly anyway to ensure that your server is up to date. the updater makes a backup of the files and database that it changes, so you can restore the old version in case that you experience problems.

    The dovecot version in debian wheezy is 2.1.7, so your server will run this version when you follow the guide.

    You just have to run the dovecot migration script that Croydon mentioned:

    http://www.howtoforge.com/forums/showpost.php?p=281080&postcount=3
     
  5. James A

    James A Member HowtoForge Supporter

    Till, many thanks for the reply and attachment.

    I'll give this a go.

    James
     
  6. James A

    James A Member HowtoForge Supporter

    Sorry Till, one last thing.

    In order to replicate the email / domain setup on the new ISPconfig I assume I only need to export certain tables from the database on the old server and write them into the same tables on the new server. When we do this on our current server to its backup we copy the following but are there any others you can think we would need. Alternatively which ones should not be copied as they hold unique data to a particular server or is there a simpler way.

    client
    client_template
    cron
    firewall
    mail_access
    mail_content_filter
    mail_domain
    mail_forwarding
    mail_get
    mail_greylist - No longer used I assume
    mail_mailman_domain - No longer used I assume
    mail_traffic
    mail_transport
    mail_user
    mail_user_filter
    spamfilter_policy
    spamfilter_users
    spamfilter_wblist
    dns_rr
    dns_soa
    dns_template

    Thanks

    James
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    Normally you would copy the whole dbispconfig database to the new server and not only some tables. The only table that holds the server specific configuration is the "server" table, but you can overwrite that as well when you do not switch to a different distribution. e.g. debian 5 > debian 7 = ok, debian 6 > centos 6.4 = not ok.
     
  8. James A

    James A Member HowtoForge Supporter

    Hi Till,

    Just wondered how I implement step 5 in the migration instruction when I'm on the latest version of ISPconfig.

    5) Run a ispconfig update and select to reconfigure services.

    At the moment I'm simply chucked out of the update process because I'm on the latest release. Do you simply edit the version number in the sys_config table in order to allow it to run again or am I missing something.

    Many thanks
    James
     
  9. till

    till Super Moderator Staff Member ISPConfig Developer

    Code:
    cd /tmp
    wget http://www.ispconfig.org/downloads/ISPConfig-3-stable.tar.gz
    tar xvfz ISPConfig-3-stable.tar.gz
    cd ispconfig3_install/install
    php -q update.php
     
  10. James A

    James A Member HowtoForge Supporter

    So close yet so far. Everything going really well and then hit an issue which I hadn't mentioned earlier.

    My current courier server setup is replicated using Glusterfs and having tried to run dovecot on a glusterfs platform it looks to be no go.

    My question is, in order to end up with a dovecot replicated server what would you do?

    - Create new courier server with all required email in it, migrate / upgrade it to dovecot then install the dsync plugin.

    - Something else?

    Thanks
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    Which problems did you have with glusterfs? e.g. dovecot runs fine on nfs, you just have to turn on a few switches (which dovecot mentions in the mail.log). So I see no reason why it shall not be possible to use glusterfs.
     
  12. James A

    James A Member HowtoForge Supporter

    Hi Till

    Thanks for your reply. The issue I had was the Gluster Client lost its connection to the cluster, it would recover after a reboot but fail as soon as I tried accessing using roundcube.

    Anyway since then I did a full rebuild and was testing roundcube access into courier prior to upgrading to dovecot. As part of the test I tried accessing a large mail client (4500 emails) through a gluster client on the gluster server and directly to non glustered var/vmail. The difference in speed was vast when applying a sort by name to the emails, 2s vs 25s per page for the glustered drive.

    Is this poor gluster performance to be expected with large numbers of files and is there a way to improve it or should I simply look for another method of replication?

    The two remote servers we have are both running gluster server and clients on the same box which I thought would have meant minimal delay but I guess I'm wrong.

    Apologies for the long post and not getting back sooner.
     
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    I had the same bad results with glusterfs in the past. glusterfs is not able to handle many small files and directories. if you use glusterfs for /var/www, then it would be even worse.

    For that reason, I switched to unison in the last ispconfig cluster tutorial, but thats not s realtime sync.

    As glusterfs has only problem with small files, it might be possible to create a loop device and put that on glusterfs, but havent tested that yet.

    another option might be to use unison together with somthing like fam to replicate files on change and not with a cronjob.

    or use the good old drbd.
     
  14. James A

    James A Member HowtoForge Supporter

    Thanks for you reply Till, unison works much better and is probably enough for us at the moment.

    Hopefully the last questions now for a while,

    - I have noticed an issue with the dovecot migration script, for us at least. It doesn't maintain the uids after upgrading. This is a major issue as our users are on very slow connections with large amounts of data on the server. Is there another script which maintains the uids or do I need to make my own hack to your script, presumably just to rename the courierimapuiddb file to dovecot-uidlist. I can also do the same with the courierimapsubscribed file, renaming it to subscriptions and removing INBOX. from infront of all subscriptions.

    -Is there any reason you keep the courierimapsubscribed folder as I thought that was the only one that couldn't migrate?

    - Finally, for some reason If I access an account from roundcube without the subscriptions file and dovecot-uidlist it appears to forget it's special folders settings, presumably because they're not in the subscriptions. What confuses me here though is I can't see where the special folder settings are stored..... of course if I follow the above method this isn't a problem but it would just be nice to know.

    Many thanks

    James
     
  15. till

    till Super Moderator Staff Member ISPConfig Developer

    There is no special reason for that, its just not implemented. if you hae some improvements for the script, then it would be great if you would post your improved version.

    No.

    I dont think that this infor is stored in the maildir. As far as I know, mailclients just serach fpr some common name. Like .Spam or .Junk for the junk folder etc. Most likely you can configure that in the roundcube config file if you dont want the names to be auto assigned.
     

Share This Page