How To Migrate Mailboxes Between IMAP Servers With imapsync

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Mon, 2009-06-01 17:50. :: Email

How To Migrate Mailboxes Between IMAP Servers With imapsync

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 05/27/2009

This guide explains how you can migrate mailboxes between IMAP servers with imapsync. imapsync is an IMAP synchronisation, sync, copy or migration tool. More than 32 different IMAP server softwares are supported with success. All flags are preserved, unread will stay unread, read will stay read, deleted will stay deleted.

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

I'm using two Debian or Ubuntu servers here (the procedure should be the same for other distributions, only the installation of imapsync will differ):

  • server1.example.com (source)
  • server2.example.com (target)

I want to migrate the mailbox sales@example.com (username: sales@example.com, password: secret) on server1.example.com to the mailbox sales@example.com on server2.example.com (username and password are the same as on server1.example.com).

 

2 Before We Start

Before we start, let's take a look at both mailboxes to see what's in them (you can do this with an email client such as Outlook, Thunderbird, or a web-based email client such as SquirrelMail - I'm using SquirrelMail here).

server1.example.com:

server2.example.com:

After the migration, the two messages on server1.example.com should be available on server2.example.com.

 

3 imapsync Installation

It doesn't matter if we install imapsync on server1.example.com or server2.example.com - we can even install it on a third, uninvolved server.

aptitude install imapsync

To learn about its usage, take a look at

man imapsync

and

imapsync --help

 

4 Doing The Migration

We have to tell imapsync the usernames and passwords of the mail accounts that we want to migrate. Theoretically, we can use the --password1 and --password2 switches for the passwords, but this is dangerous because the passwords would appear in the output of

ps aux

(So any user who is allowed to run that command would know the passwords.)

Therefore it's better to create two files (/etc/secret1 and /etc/secret2 in this example) that contain the passwords (in this example, it's secret for both mail accounts), do a chmod 600 on these files, and use the --passfile1 and --passfile2 switches.

echo "secret" > /etc/secret1
echo "secret" > /etc/secret2
chmod 600 /etc/secret1
chmod 600 /etc/secret2

Now we can do the migration with this simple command:

imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2

Here's a sample output:

root@server1:~# imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2
 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2
$RCSfile: imapsync,v $ $Revision: 1.252 $ $Date: 2008/05/08 02:30:17 $
Here is a [linux] system (Linux server1.example.com 2.6.28-11-server #42-Ubuntu SMP Fri Apr 17 02:45:36 UTC 2009 x86_64)
with perl 5.10.0 and the module Mail::IMAPClient version used here is 2.2.9
Command line used :
/usr/bin/imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2
 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2
Turned ON syncinternaldates, will set the internal dates on host2 same as host1.
TimeZone :[CEST]
Will try to use CRAM-MD5 authentication on host1
Will try to use CRAM-MD5 authentication on host2
From imap server [server1.example.com] port [143] user [sales@example.com]
To   imap server [server2.example.com] port [143] user [sales@example.com]
Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA
 IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc.  See COPYING for distribution information.
Host server1.example.com says it has NO CAPABILITY for AUTHENTICATE CRAM-MD5
Error login : [server1.example.com] with user [sales@example.com] auth [CRAM-MD5]: 3 NO Login failed.

Trying LOGIN Auth mechanism on [server1.example.com] with user [sales@example.com]
Success login on [server1.example.com] with user [sales@example.com] auth [CRAM-MD5]
Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA
 IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc.  See COPYING for distribution information.
Host server2.example.com says it has NO CAPABILITY for AUTHENTICATE CRAM-MD5
Error login : [server2.example.com] with user [sales@example.com] auth [CRAM-MD5]: 3 NO Login failed.

Trying LOGIN Auth mechanism on [server2.example.com] with user [sales@example.com]
Success login on [server2.example.com] with user [sales@example.com] auth [CRAM-MD5]
From capability : QUOTA STARTTLS NAMESPACE IDLE THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN ACL2=UNION IMAP4REV1 THREAD=REFERENCES
To   capability : QUOTA STARTTLS NAMESPACE IDLE THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN ACL2=UNION IMAP4REV1 THREAD=REFERENCES
From state Authenticated
To   state Authenticated
From separator and prefix : [.][INBOX.]
To   separator and prefix : [.][INBOX.]
++++ Calculating sizes ++++
From Folder [INBOX]                             Size:      2412 Messages:     2
From Folder [INBOX.Drafts]                      Size:         0 Messages:     0
From Folder [INBOX.Sent]                        Size:         0 Messages:     0
From Folder [INBOX.Trash]                       Size:         0 Messages:     0
Total size: 2412
Total messages: 2
Time : 11 s
++++ Calculating sizes ++++
To   Folder [INBOX]                             Size:     16844 Messages:     6
To   Folder [INBOX.Drafts]                      Size:         0 Messages:     0
To   Folder [INBOX.Sent]                        Size:         0 Messages:     0
To   Folder [INBOX.Trash]                       Size:         0 Messages:     0
Total size: 16844
Total messages: 6
Time : 0 s
++++ Listing folders ++++
From folders list : [INBOX] [INBOX.Drafts] [INBOX.Sent] [INBOX.Trash]
To   folders list : [INBOX] [INBOX.Drafts] [INBOX.Sent] [INBOX.Trash]
++++ Looping on each folder ++++
From Folder [INBOX]
To   Folder [INBOX]
++++ From [INBOX] Parse 1 ++++
++++ To   [INBOX] Parse 1 ++++
++++ Verifying [INBOX] -> [INBOX] ++++
+ NO msg #1 [FID6mR7SVC7csdmRce+7PQ:924] in INBOX
+ Copying msg #1:924 to folder INBOX
flags from : [\Seen]["26-May-2009 22:00:59 +0200"]
Copied msg id [1] to folder INBOX msg id [7]
+ NO msg #2 [4lu3SCjZGH2mRuiP1dY7Mw:1488] in INBOX
+ Copying msg #2:1488 to folder INBOX
flags from : [()]["27-May-2009 14:12:59 +0200"]
Copied msg id [2] to folder INBOX msg id [8]
Time : 0 s
From Folder [INBOX.Drafts]
To   Folder [INBOX.Drafts]
++++ From [INBOX.Drafts] Parse 1 ++++
++++ To   [INBOX.Drafts] Parse 1 ++++
++++ Verifying [INBOX.Drafts] -> [INBOX.Drafts] ++++
Time : 0 s
From Folder [INBOX.Sent]
To   Folder [INBOX.Sent]
++++ From [INBOX.Sent] Parse 1 ++++
++++ To   [INBOX.Sent] Parse 1 ++++
++++ Verifying [INBOX.Sent] -> [INBOX.Sent] ++++
Time : 0 s
From Folder [INBOX.Trash]
To   Folder [INBOX.Trash]
++++ From [INBOX.Trash] Parse 1 ++++
++++ To   [INBOX.Trash] Parse 1 ++++
++++ Verifying [INBOX.Trash] -> [INBOX.Trash] ++++
Time : 1 s
++++ Statistics ++++
Time                   : 12 sec
Messages transferred   : 2
Messages skipped       : 0
Total bytes transferred: 2412
Total bytes skipped    : 0
Total bytes error      : 0
Detected 0 errors
Please, rate imapsync at http://freshmeat.net/projects/imapsync/
?Happy with this free, open source and gratis GPL software?
Feel free to thank the author by giving him a book:
http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
(or its paypal account gilles.lamiral@laposte.net)
root@server1:~#

Afterwards we can delete the two password files:

rm -f /etc/secret1
rm -f /etc/secret2

On the target server (server2.example.com), log out of your IMAP account and log back in (a simple refresh might not be enough), and you should see the messages from server1.example.com:

server2.example.com:

 

5 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Albert (not registered) on Fri, 2014-04-11 07:45.

This worked beautifully, and it posted with a security conscience.

 Thank you so much!!! - albertautin.com

Submitted by Anonymous (not registered) on Tue, 2012-03-13 04:06.
If we don't know any password from users, can admin password or the account with admin right help with the migration???

 

Submitted by Debapriya Biswas (not registered) on Fri, 2012-01-20 05:50.
We are trying to migrate our Lotus Domino 8.5 server to Zimbra Collaboration Suite. Can we use IMAPSYNC to do the task? 

Do we need to provide all username & passwords we want to migrate from Lotus (Server1) to Zimbra (Server2)?

Is there any way to do the migration with ADMIN Accounts on both servers? 

 Please advise.

Submitted by excalibur74 (not registered) on Tue, 2011-04-12 14:03.

Hi, how are you?

one question, this tool is for mailbox to maildir too? i need migrate all mailbox´s (in opensuse) to another server with debian and maildir format.

thanks very much

(sorry my english)

bye bye

Submitted by Daniel Plaenitz (not registered) on Sun, 2010-12-05 11:29.

Thanks for the easy to follow instructions, however, it won't do IPv6.
Error connecting /  IO::Socket::SSL: Bad hostname.
The dire state of perl ...

 

Submitted by Jacob (not registered) on Fri, 2010-11-05 17:27.

Thank you Falko for your clear instructions! I got imapsync working just fine, but realized it only syncs one way, not both ways. I have tried the following solutions:

- OfflineIMAP 4.0.16: works (fast!!!) both ways but doesn't sync new folders, crashes on folder deletion 
- imapsync: works great but only one way, server1 -> server2, that's it
- isync: doesn't sync IMAP <-> IMAP, only IMAP <-> Maildir

 I'm about to give up on my project - trying to keep the 60 IMAP mailboxes/20 GB at our provider synchronized with a local NetBSD server running Dovecot in order to improve access time and backup in case our internet connection goes out. I want users to access our local IMAP server as if they were accessing our ISP's server and never know the difference. Our ISP has a good webmail server (SurgeMail 5) and I want to keep the remote boxes identical to the local ones for this reason.

 So far, nothing worked out simply because I need something that seems to be unavailable at the moment. Does anyone know about a solution that I can use on my NetBSD installation?

Submitted by Fran Martin (not registered) on Wed, 2010-10-27 00:49.

I think that's a quite painful process for a regular user. Instead, you could give a try to our tool, it's fast, secure and reliable and very easy to use since you don't need to download anything.

It also keeps label structure and transfer all your email in seconds

Improffice Email Transfer - Gmail and Google Apps Migrations

Submitted by Andrew (not registered) on Wed, 2010-06-09 22:18.

There's a typo in the imapsync commands passfile1 string

imapsync --host1 server1.example.com --user1 sales@example.com --passfile1 /etc/secret2 --host2 server2.example.com --user2 sales@example.com --passfile2 /etc/secret2


it should read "passfile1 /etc/secret1" instead of "/etc/secret2"


Submitted by Anonymous (not registered) on Thu, 2010-04-22 11:51.

Hi,

Can we use imapsync to synchonize two different domains eg. from example.com to example1.com.

 

Thanks and Regards

Imran

 

Submitted by Br8knitOFF (not registered) on Sat, 2010-03-27 17:02.

Falko,

Do you know if imapsync will work with a Maildir that is copied to a locally attached portable disk and then attached to the new target server?

Reason I ask is because we have about 40GB of mail to migrate to a new server over ISDN.

If not, do you know of a better way with that locally copied mail brought down via that portable HDD?

Thanks!

//Todd

Submitted by Anonymous (not registered) on Mon, 2010-01-25 18:30.

Great guide. I've used imapsync a lot in the past, but I find it very resource intense (in particular if you're moving lots of mailboxes). Moreover, you're limited to the local bandwidth of the server running imapsync.

 There are some good alternatives though, such as Yippiemove (Any IMAP-to-IMAP migration), Google Mail Migration (any IMAP to Google Apps Premier/Edu) and Rackspace Migration (IMAP and MAPI to Rackspace). All of those are fast, real easy to use, and do not require you to get you hands dirty.

Submitted by Ryan Kulp (not registered) on Mon, 2013-07-22 21:49.

Another great one is ShuttleCloud's email migration tool.

It's completely free on the Google Apps Marketplace, with no strings attached or trial versions. 

Submitted by Ryan Kulp (not registered) on Thu, 2013-08-08 22:20.

ShuttleCloud also offers a version of the tool that doesn't require a Google Apps account.

 The tool is called <a href="TransferMyEmail.com">TransferMyEmail</a>

Submitted by Nisha Kumar (not registered) on Mon, 2010-01-04 13:29.

Appreciate the time and effort put forth in putting this out there. I know a lot of people who would find this very informative to say the least.

 Thanks,

Nisha

Submitted by Johan Andersson (not registered) on Fri, 2009-11-27 18:01.

This tool seems quite good, the inly thing I see lacking is the ability to map certain from targets to certain to targets... i.e. the

Sent -> Sent Items

 

 

Submitted by Uwe Brauer (not registered) on Fri, 2009-06-05 14:47.

Hello

 thanks that looks interesting. I am running a dovecot imap server but use the python script offlineimap, maybe imapsync is faster

 

Uwe Brauer  

Submitted by Thiago Ferreira (not registered) on Thu, 2009-06-04 16:15.
I use "Dovecot",  how I do to migrate all the mailboxes to another server without using "rsync", whith the command "imapsync" ?
I like to migrate but without to use logins and passwords...
 
Thanks all.
Thiago.
Submitted by David Ambrose-G... (not registered) on Thu, 2009-06-18 16:41.

You would need to configure your dovecot server to work with an Admin username and password as well as the regular user's usernames and passwords.

You can then use the --authuser1 --authuser2 flags to imapsync to specify a user to login as (which is different from the user who's mail your syncing)

http://wiki.dovecot.org/Authentication/MasterUsers

 

Submitted by ecommerce guy (not registered) on Tue, 2009-06-02 23:49.
Its a good technique, i found it easy to deal with .. thanks inowweb.com