Moving ISPConfig from single mailfolder to Maildir

Discussion in 'Tips/Tricks/Mods' started by Morons, Feb 21, 2007.

  1. Morons

    Morons Member

    Hi guys, Big fun
    I started to convert from Mandriva to Ubuntu and have set-up both these using the "Perfect set-up" however the native pop3d on Mandriva authenticate via pam and retrieve mail from folder /var/spool/mail/username. On Ubuntu the courier suite is used and the pop3d can not work unless against Maildir setup.

    The Ubuntu setup was The Perfect Setup - Ubuntu 6.10 Server (Edgy Eft) Page 5 and
    the Move instruction used many times between servers was Moving ISPConfig I also added the FuzzyOCR, Pyzor, Ryzor, DCC, Postgrey as per this forum instructions. All tests passed!

    The problem is that I can find how-to’s on converting to Maildir on Mandriva but it discard the "old" mail (couple Gig's) and start fresh. I can not use fetchmail either as the passwords are MD5 or otherwise encrypted, changed by user etc. ( I have not investigated the possibility to run fetchmail with those encrypted passwords). I presume I could change all passwords to one I know and afterwards replace the passwd/shadow file from backup -- hmmm would need to think about that one.

    Any case I'm stuck, If anyone have answers to quick solving this without causing long downtime - please let me know. Normally the above move take only 5 minutes to backup and restore the data once the replacement server are set-up, that is not a great deal of downtime.

    I think the best method still would be to 1st convert to Maildir BUT that has to migrate old mail content also.
    Last edited: Feb 21, 2007
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    IEither you convert your mailboxes with the mb2md script (which has worked fine for me some time ago) or you install a pop3 and imap daemon on ubuntu that supports mailbox.
  3. Morons

    Morons Member

    mb2md hmmm, will have a lookcee, BTW this was really fast reply while I edited my grammar and within 6 minutes!

    Thank you Till:)
  4. Morons

    Morons Member

    Converting mailbox to maildir via scripts

    My solution I used was to move all the users and mail over to the new ubuntu box using courier pop3d with Maildir as if it used mailbox and then convert the mail to Maildir leaving me with an nice working system. the scripts I used was a bit of an mix and match of scripts - Plainly because the manpages for mb2md was as clear as mud for me, I do not know why the manpage writers can't write the darn stuff in plain english! :mad:
    I started of my hacking the script that cut the passwd file in sections used in this forum by the process of making the userdb file for [email protected] login.
    Make a file called /usr/bin/2mb with the following in it.
    for line in `/usr/sbin/pw2userdb | grep -ir user`
            full=$( echo $line )
            domain=$( echo $line | cut -d"$delim" -f1 )
            remain=$( echo $line | cut -d"$delim" -f2,3,4 )
            username=$( echo $remain | cut -d'      ' -f1 )
            remain2=$( echo $remain | cut -d'       ' -f2 )
            uid=$( echo $remain2 | cut -d'|' -f1 )
            gid=$( echo $remain2 | cut -d'|' -f2 )
            home=$( echo $remain2 | cut -d'|' -f3 )
            hm=$( echo $home | cut -c 6- )
            /usr/bin/maildirmake $hm/Maildir
            /usr/bin/perfect_maildir $hm/Maildir < $username
            chown -R $username:mail $hm/Maildir
    Note the spaces you will get if you copy this to the files I called /usr/bin/2mb is actually tabs, one single tab. those lines are
            username=$( echo $remain | cut -d'      ' -f1 )
            remain2=$( echo $remain | cut -d'       ' -f2 )
    The main core work is done by an script i downloaded (by Philip Mak) and this lives in /usr/bin/perfect_maildir
    # "Simple but Perfect" mbox to Maildir converter v0.1
    # by Philip Mak <[EMAIL PROTECTED]>
    # Usage: perfect_maildir ~/Maildir < mbox
    # Simple  - only converts one mbox (can use script in one-liners)
    # Perfect - message Flags/X-Flags are converted; "^>From ." line is unescaped
    # I wrote this script after being unsatisfied with existing mbox to
    # maildir converters. By making it "Simple", code complexity is kept
    # low thus making it easy to program and debug. At the same time,
    # since it only converts one mbox at a time, it is perfect for use in
    # a shell ``for'' loop (for example).
    # As for being "Perfect", to the best of my knowledge this script does
    # the conversion correctly in all cases; it will translate "Status"
    # and "X-Status" fields into maildir info, and it correctly detects
    # where messages begin and end. (This is only version 0.1 so I may
    # have messed something up though. Please send me feedback!)
    # NOTE: The MUA ``mutt'' has a bug/feature where in the message index,
    # it claims that all maildir messages have 0 lines unless they have a
    # "Lines:" header set. perfect_maildir does not attempt to add the
    # "Lines:" header; you may want to reconfigure ``mutt' to display byte
    # size instead of lines instead by adding the following line to your
    # ~/.muttrc file:
    # set index_format="%4C %Z %{%b %d} %-15.15L (%4c) %s"
    # check for valid arguments
    my ($maildir) = @ARGV;
    if (!$maildir) {
      print STDERR "Usage: perfect_maildir ~/Maildir < mbox\n";
      exit 1;
    # check for writable maildir
    unless (-w "$maildir/cur") {
      print STDERR "Cannot write to $maildir/cur\n";
      exit 1;
    unless (-w "$maildir/new") {
      print STDERR "Cannot write to $maildir/new\n";
      exit 1;
    my $num = 0;
    my $time = time;
    # read header
    my $headers = '';
    my $flags = '';
    my $subject = '';
    while (my $line = <STDIN>) {
      # detect end of headers
      last if $line eq "\n";
      # strip "From" line from header
      $headers .= $line unless $line =~ /^From ./;
      # detect flags
      $flags .= $1 if $line =~ /^Status: ([A-Z]+)/;
      $flags .= $1 if $line =~ /^X-Status: ([A-Z]+)/;
      $subject = $1 if $line =~ /^Subject: (.*)$/;
    # open output file
    my $file;
    if ($flags =~ /O/) {
      $file = "$maildir/cur/$time.$num.$ENV{HOSTNAME}";
      my $extra = '';
      $extra .= 'F' if $flags =~ /F/; # flagged
      $extra .= 'R' if $flags =~ /A/; # replied
      $extra .= 'S' if $flags =~ /R/; # seen
      $extra .= 'T' if $flags =~ /D/; # trashed
      $file .= ":2,$extra" if $extra;
    } else {
      $file = "$maildir/new/$time.$num.$ENV{HOSTNAME}";
    # filter out the "DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA" message
    $file = '/dev/null' if ($num == 1 and $subject eq "DON'T DELETE THIS MESSAGE -- FOLDER
    open(FILE, ">$file");
    print FILE "$headers\n";
    while (my $line = <STDIN>) {
      # detect end of message
      last if $line =~ /^From ./;
      # unescape "From"
      $line =~ s/^>From (.)/From $1/;
      print FILE $line;
    goto repeat unless eof(STDIN);
    my $elapsed = time - $time;
    print "Inserted $num messages into maildir $maildir in $elapsed seconds\n";
    Make sure to make the files executable
    chmod +x /usr/bin/perfect_maildir
    chmod +x /usr/bin/2mb
    EDIT: You need to be in the directory that contains the mailbox files else 2mb gives an error! The line in the /usr/bin/2mb that causes the error is
    /usr/bin/perfect_maildir $hm/Maildir < $username
    As the last reference to < $username does not include an path!

    Works like an charm!:)
    Last edited: Feb 23, 2007
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Thanks for posting your solution :) I moved the thread to the Tipps & Tricks forum.
  6. blocker

    blocker Member


    thank you for the clever trick :)

    i have a problem on implementing it on one of my servers. The problem is that it gives back an error when converting mbox to Maildir of a user which user name contains a "." (dot).

    any ideas how can i work around this issue?


Share This Page