How To Migrate Mailboxes Between IMAP Servers With imapsync

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Submitted by falko (Contact Author) (Forums) on Mon, 2009-06-01 17:50. :: Email

How To Migrate Mailboxes Between IMAP Servers With imapsync

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 05/27/2009

This guide explains how you can migrate mailboxes between IMAP servers with imapsync. imapsync is an IMAP synchronisation, sync, copy or migration tool. More than 32 different IMAP server softwares are supported with success. All flags are preserved, unread will stay unread, read will stay read, deleted will stay deleted.

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

I'm using two Debian or Ubuntu servers here (the procedure should be the same for other distributions, only the installation of imapsync will differ):

  • server1.example.com (source)
  • server2.example.com (target)

I want to migrate the mailbox sales@example.com (username: sales@example.com, password: secret) on server1.example.com to the mailbox sales@example.com on server2.example.com (username and password are the same as on server1.example.com).

 

2 Before We Start

Before we start, let's take a look at both mailboxes to see what's in them (you can do this with an email client such as Outlook, Thunderbird, or a web-based email client such as SquirrelMail - I'm using SquirrelMail here).

server1.example.com:

server2.example.com:

After the migration, the two messages on server1.example.com should be available on server2.example.com.

 

3 imapsync Installation

It doesn't matter if we install imapsync on server1.example.com or server2.example.com - we can even install it on a third, uninvolved server.

aptitude install imapsync

To learn about its usage, take a look at

man imapsync

and

imapsync --help

 

4 Doing The Migration

We have to tell imapsync the usernames and passwords of the mail accounts that we want to migrate. Theoretically, we can use the --password1 and --password2 switches for the passwords, but this is dangerous because the passwords would appear in the output of

ps aux

(So any user who is allowed to run that command would know the passwords.)

Therefore it's better to create two files (/etc/secret1 and /etc/secret2 in this example) that contain the passwords (in this example, it's secret for both mail accounts), do a chmod 600 on these files, and use the --passfile1 and --passfile2 switches.

echo "secret" > /etc/secret1
echo "secret" > /etc/secret2
chmod 600 /etc/secret1
chmod 600 /etc/secret2

Now we can do the migration with this simple command:

imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2

Here's a sample output:

root@server1:~# imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2
 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2
$RCSfile: imapsync,v $ $Revision: 1.252 $ $Date: 2008/05/08 02:30:17 $
Here is a [linux] system (Linux server1.example.com 2.6.28-11-server #42-Ubuntu SMP Fri Apr 17 02:45:36 UTC 2009 x86_64)
with perl 5.10.0 and the module Mail::IMAPClient version used here is 2.2.9
Command line used :
/usr/bin/imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2
 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2
Turned ON syncinternaldates, will set the internal dates on host2 same as host1.
TimeZone :[CEST]
Will try to use CRAM-MD5 authentication on host1
Will try to use CRAM-MD5 authentication on host2
From imap server [server1.example.com] port [143] user [sales@example.com]
To   imap server [server2.example.com] port [143] user [sales@example.com]
Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA
 IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc.  See COPYING for distribution information.
Host server1.example.com says it has NO CAPABILITY for AUTHENTICATE CRAM-MD5
Error login : [server1.example.com] with user [sales@example.com] auth [CRAM-MD5]: 3 NO Login failed.

Trying LOGIN Auth mechanism on [server1.example.com] with user [sales@example.com]
Success login on [server1.example.com] with user [sales@example.com] auth [CRAM-MD5]
Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA
 IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc.  See COPYING for distribution information.
Host server2.example.com says it has NO CAPABILITY for AUTHENTICATE CRAM-MD5
Error login : [server2.example.com] with user [sales@example.com] auth [CRAM-MD5]: 3 NO Login failed.

Trying LOGIN Auth mechanism on [server2.example.com] with user [sales@example.com]
Success login on [server2.example.com] with user [sales@example.com] auth [CRAM-MD5]
From capability : QUOTA STARTTLS NAMESPACE IDLE THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN ACL2=UNION IMAP4REV1 THREAD=REFERENCES
To   capability : QUOTA STARTTLS NAMESPACE IDLE THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN ACL2=UNION IMAP4REV1 THREAD=REFERENCES
From state Authenticated
To   state Authenticated
From separator and prefix : [.][INBOX.]
To   separator and prefix : [.][INBOX.]
++++ Calculating sizes ++++
From Folder [INBOX]                             Size:      2412 Messages:     2
From Folder [INBOX.Drafts]                      Size:         0 Messages:     0
From Folder [INBOX.Sent]                        Size:         0 Messages:     0
From Folder [INBOX.Trash]                       Size:         0 Messages:     0
Total size: 2412
Total messages: 2
Time : 11 s
++++ Calculating sizes ++++
To   Folder [INBOX]                             Size:     16844 Messages:     6
To   Folder [INBOX.Drafts]                      Size:         0 Messages:     0
To   Folder [INBOX.Sent]                        Size:         0 Messages:     0
To   Folder [INBOX.Trash]                       Size:         0 Messages:     0
Total size: 16844
Total messages: 6
Time : 0 s
++++ Listing folders ++++
From folders list : [INBOX] [INBOX.Drafts] [INBOX.Sent] [INBOX.Trash]
To   folders list : [INBOX] [INBOX.Drafts] [INBOX.Sent] [INBOX.Trash]
++++ Looping on each folder ++++
From Folder [INBOX]
To   Folder [INBOX]
++++ From [INBOX] Parse 1 ++++
++++ To   [INBOX] Parse 1 ++++
++++ Verifying [INBOX] -> [INBOX] ++++
+ NO msg #1 [FID6mR7SVC7csdmRce+7PQ:924] in INBOX
+ Copying msg #1:924 to folder INBOX
flags from : [\Seen]["26-May-2009 22:00:59 +0200"]
Copied msg id [1] to folder INBOX msg id [7]
+ NO msg #2 [4lu3SCjZGH2mRuiP1dY7Mw:1488] in INBOX
+ Copying msg #2:1488 to folder INBOX
flags from : [()]["27-May-2009 14:12:59 +0200"]
Copied msg id [2] to folder INBOX msg id [8]
Time : 0 s
From Folder [INBOX.Drafts]
To   Folder [INBOX.Drafts]
++++ From [INBOX.Drafts] Parse 1 ++++
++++ To   [INBOX.Drafts] Parse 1 ++++
++++ Verifying [INBOX.Drafts] -> [INBOX.Drafts] ++++
Time : 0 s
From Folder [INBOX.Sent]
To   Folder [INBOX.Sent]
++++ From [INBOX.Sent] Parse 1 ++++
++++ To   [INBOX.Sent] Parse 1 ++++
++++ Verifying [INBOX.Sent] -> [INBOX.Sent] ++++
Time : 0 s
From Folder [INBOX.Trash]
To   Folder [INBOX.Trash]
++++ From [INBOX.Trash] Parse 1 ++++
++++ To   [INBOX.Trash] Parse 1 ++++
++++ Verifying [INBOX.Trash] -> [INBOX.Trash] ++++
Time : 1 s
++++ Statistics ++++
Time                   : 12 sec
Messages transferred   : 2
Messages skipped       : 0
Total bytes transferred: 2412
Total bytes skipped    : 0
Total bytes error      : 0
Detected 0 errors
Please, rate imapsync at http://freshmeat.net/projects/imapsync/
?Happy with this free, open source and gratis GPL software?
Feel free to thank the author by giving him a book:
http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
(or its paypal account gilles.lamiral@laposte.net)
root@server1:~#

Afterwards we can delete the two password files:

rm -f /etc/secret1
rm -f /etc/secret2

On the target server (server2.example.com), log out of your IMAP account and log back in (a simple refresh might not be enough), and you should see the messages from server1.example.com:

server2.example.com:

 

5 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Andrew (not registered) on Wed, 2010-06-09 22:18.

There's a typo in the imapsync commands passfile1 string

imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2


it should read "passfile1 /etc/secret1" instead of "/etc/secret2"


Submitted by Anonymous (not registered) on Thu, 2010-04-22 11:51.

Hi,

Can we use imapsync to synchonize two different domains eg. from example.com to example1.com.

 

Thanks and Regards

Imran

 

Submitted by Br8knitOFF (not registered) on Sat, 2010-03-27 17:02.

Falko,

Do you know if imapsync will work with a Maildir that is copied to a locally attached portable disk and then attached to the new target server?

Reason I ask is because we have about 40GB of mail to migrate to a new server over ISDN.

If not, do you know of a better way with that locally copied mail brought down via that portable HDD?

Thanks!

//Todd

Submitted by Anonymous (not registered) on Mon, 2010-01-25 18:30.

Great guide. I've used imapsync a lot in the past, but I find it very resource intense (in particular if you're moving lots of mailboxes). Moreover, you're limited to the local bandwidth of the server running imapsync.

 There are some good alternatives though, such as Yippiemove (Any IMAP-to-IMAP migration), Google Mail Migration (any IMAP to Google Apps Premier/Edu) and Rackspace Migration (IMAP and MAPI to Rackspace). All of those are fast, real easy to use, and do not require you to get you hands dirty.

Submitted by Nisha Kumar (not registered) on Mon, 2010-01-04 13:29.

Appreciate the time and effort put forth in putting this out there. I know a lot of people who would find this very informative to say the least.

 Thanks,

Nisha

Submitted by Johan Andersson (not registered) on Fri, 2009-11-27 18:01.

This tool seems quite good, the inly thing I see lacking is the ability to map certain from targets to certain to targets... i.e. the

Sent -> Sent Items

 

 

Submitted by Uwe Brauer (not registered) on Fri, 2009-06-05 14:47.

Hello

 thanks that looks interesting. I am running a dovecot imap server but use the python script offlineimap, maybe imapsync is faster

 

Uwe Brauer  

Submitted by Thiago Ferreira (not registered) on Thu, 2009-06-04 16:15.
I use "Dovecot",  how I do to migrate all the mailboxes to another server without using "rsync", whith the command "imapsync" ?
I like to migrate but without to use logins and passwords...
 
Thanks all.
Thiago.
Submitted by David Ambrose-G... (not registered) on Thu, 2009-06-18 16:41.

You would need to configure your dovecot server to work with an Admin username and password as well as the regular user's usernames and passwords.

You can then use the --authuser1 --authuser2 flags to imapsync to specify a user to login as (which is different from the user who's mail your syncing)

http://wiki.dovecot.org/Authentication/MasterUsers

 

Submitted by ecommerce guy (not registered) on Tue, 2009-06-02 23:49.
Its a good technique, i found it easy to deal with .. thanks inowweb.com
Sponsored Links: Turn your desk phone and mobile phone into one with Sprint Mobile Integration.
www.seamlessenterprise.com

One number. One voicemail. Seize the lead. Sprint Mobile Integration.
www.seamlessenterprise.com

One Number. One Voicemail.
Make it easier for clients to reach you. Turn your desk phone and mobile phone into one with Sprint Mobile Integration.
www.seamlessenterprise.com

One number. One voicemail. Sprint Mobile Integration.
www.seamlessenterprise.com

One number. one voicemail. Seize the lead with Sprint. Learn more

AT&T Synaptic Compute as a Service. Boost your power on demand.

Trial: IBM Cognos Express Reporting, Analysis & Planning

Learn benefits of Simpana software.
View the Gartner Video

Sprint 4G - The Ultimate Mobile Broadband
Click here

SAP-Business Objects Crystal Reports Server
Complete reporting without hidden costs. Free Trial