email archiving

[Ovidiu]

I've searched and found a couple of threads here asking the same question but no proper solution. Mostly these threads were a few years old so I am starting my own here.

Basically I have clients hosting their emails with me that need to comply with certain regional laws which means 100% reliable storage of all incoming/outgoing emails. Lets not worry about how I back this up, for now I only need a solution for archiving.

My base system runs ISPCFG3 with Debian Squeeze.

Needed features:

• archive all emails, incoming and outgoing on a per email domain basis

Nice to have features:

• web interface for searching and restoring from the archive into the original location
• CLI for restoring from the archive into the original location

What I have found so far:

• easy postfix solution, need more explanations on this one: http://www.arschkrebs.de/postfix/postfix_archive.shtml
• DBMail - not sure this is what I need, seems to be a complete IMAP server? http://www.dbmail.org/
• Commercial solution, not sure if it is still being developed, need to contact them: http://xamime.com/
• Milters, never worked with them, this is the archiving section: https://www.milter.org/milters/archiving/alphabetical/1

To be honest, the seemingly easiest solution from the ones I listed above seems to be this one: http://www.arschkrebs.de/postfix/postfix_archive.shtml but I admit I don't completely understand it.

Does anyone here have a similar solution in place and is willing to share the details?

[dipeshmehta]

you may use sender_bcc_maps and recipient_bcc_maps feature.

For example: if you want bcc of all outgoing message, you may create a file with any name in /etc/postfix with something like this:

*@domain1.com mailaddress@domain1.com
*@domain2.com mailaddress@domain2.com

then run

postmap /etc/postfix/filename

and
add following line to /etc/postfix/main.cf

sender_bcc_maps = hash:/etc/postfix/filename

Similar for recipient_bcc_maps.

then, restart postfix

You may find details at http://www.postfix.org/postconf.5.html

[Ovidiu]

Thanks, that looks like the other postfix URL I posted except you explained it better :-)
Will read through your link, give it a try and report back :-)

[Ovidiu]

I think I already need help :-)

lets take sender_bcc_maps as an example:

lets say I use:
*@domain1.com mailaddress@domain1.com

to collect a copy of all outgoing emails at that address. would that work even if I set that email address that collects all BCCs as inactive in ISPCFG3? I'd like to have it inactive as to not receive "normal" emails, would this postfix internal method still work?

The documentation says:

The BCC address (multiple results are not supported)

- what exactly do they mean by that?

[dipeshmehta]

Originally Posted by Ovidiu

lets say I use:
*@domain1.com mailaddress@domain1.com

to collect a copy of all outgoing emails at that address. would that work even if I set that email address that collects all BCCs as inactive in ISPCFG3?

I am not using ISPC, so I am little unfamiliar with its config. Well, the suggested method is a feature of postfix, and it forwards bcc to the given address which is normal user account as others.[/QUOTE]

Originally Posted by Ovidiu

I'd like to have it inactive as to not receive "normal" emails, would this postfix internal method still work?

you may create recipient account with some random string, to minimize chances of receiving normal mails. (e.g. sentlog.hf566412z@domain.com)

[Ovidiu]

I need to add a small correction to the above solution, you don't need

*

, it doesn't even work. Skip it and all is good:

@domain1.com mailaddress@domain1.com

Regarding other solutions, I contacted the author of one of the milters for archiving via psotfix: http://www.dancingfortune.com/projec...smtp/index.php

and he explained the advantage of using the milter vs sender_bcc_maps/recipient_bcc_maps as follows:

One of the main issues with archiving SENT mail is that the user can essentially put any From: address they like. What we do to solve this problem is require that users must provide login credentials to send mail via smtp_auth. In this case the milter is able to see the username of the sender and determine where to archive it, regardless of what From: address is in the header. This is a case worth considering if the client requires the archive for legal reasons and not just backup purposes.

Sounds right to me, I think this is a more "proper" solution, am I right?

[dipeshmehta]

Originally Posted by Ovidiu

One of the main issues with archiving SENT mail is that the user can essentially put any From: address they like. What we do to solve this problem is require that users must provide login credentials to send mail via smtp_auth. In this case the milter is able to see the username of the sender and determine where to archive it, regardless of what From: address is in the header. This is a case worth considering if the client requires the archive for legal reasons and not just backup purposes.

Sounds right to me, I think this is a more "proper" solution, am I right?

Sounds right to me too. Go ahead and post your experience here, all the best to you

[Ovidiu]

Thanks, will reply here if I reach a working config, that milter doesn't come with debian instructions so I am battling with it...

[till]

If you use a courier setup then you can archive the mails with a maildrop script, just add a rule to /var/vmail/.mailfilter to store a copy of all emails to a separate directory before delivering them to the mailbox.

If you use dovecot then you can not store the emails to a folder directly as sive does not support that but you can forward them to a technical "archiving" address and also set flags in the mail header if you want to kep the original recipient address etc.

[Ovidiu]

Thanks, that is all great information but I realized one would need something else for a "proper" legal email archive so I had a look at open source systems but the only one I could find is enkive so I suggested it as a HowTo here: http://www.howtoforge.com/forums/showthread.php?t=57215