Virtual Users And Domains With Postfix, Courier, MySQL And SquirrelMail (Ubuntu 12.04 LTS) - Page 3

9 Install amavisd-new, SpamAssassin, And ClamAV

To install amavisd-new, spamassassin and clamav, run the following command:

apt-get install amavisd-new spamassassin clamav clamav-daemon zoo unzip bzip2 libnet-ph-perl libnet-snpp-perl libnet-telnet-perl nomarch lzop pax

Afterwards we must configure amavisd-new. The configuration is split up in various files which reside in the /etc/amavis/conf.d directory. Take a look at each of them to become familiar with the configuration. Most settings are fine, however we must modify three files:

First we must enable ClamAV and SpamAssassin in /etc/amavis/conf.d/15-content_filter_mode by uncommenting the @bypass_virus_checks_maps and the @bypass_spam_checks_maps lines:

vi /etc/amavis/conf.d/15-content_filter_mode

The file should look like this:

use strict;

# You can modify this file to re-enable SPAM checking through spamassassin
# and to re-enable antivirus checking.

#
# Default antivirus checking mode
# Please note, that anti-virus checking is DISABLED by
# default.
# If You wish to enable it, please uncomment the following lines:


@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);


#
# Default SPAM checking mode
# Please note, that anti-spam checking is DISABLED by
# default.
# If You wish to enable it, please uncomment the following lines:


@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

1;  # ensure a defined return

And then you should take a look at the spam settings and the actions for spam-/virus-mails in /etc/amavis/conf.d/20-debian_defaults. There's no need to change anything if the default settings are ok for you. The file contains many explanations so there's no need to explain the settings here:

vi /etc/amavis/conf.d/20-debian_defaults

[...]
$QUARANTINEDIR = "$MYHOME/virusmails";
$quarantine_subdir_levels = 1; # enable quarantine dir hashing

$log_recip_templ = undef;    # disable by-recipient level-0 log entries
$DO_SYSLOG = 1;              # log via syslogd (preferred)
$syslog_ident = 'amavis';    # syslog ident tag, prepended to all messages
$syslog_facility = 'mail';
$syslog_priority = 'debug';  # switch to info to drop debug output, etc

$enable_db = 1;              # enable use of BerkeleyDB/libdb (SNMP and nanny)
$enable_global_cache = 1;    # enable use of libdb-based cache if $enable_db=1

$inet_socket_port = 10024;   # default listening socket

$sa_spam_subject_tag = '***SPAM*** ';
$sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.31; # triggers spam evasive actions
$sa_dsn_cutoff_level = 10;   # spam level beyond which a DSN is not sent
[...]
$final_virus_destiny      = D_DISCARD;  # (data not lost, see virus quarantine)
$final_banned_destiny     = D_BOUNCE;   # D_REJECT when front-end MTA
$final_spam_destiny       = D_BOUNCE;
$final_bad_header_destiny = D_PASS;     # False-positive prone (for spam)
[...]

Finally, edit /etc/amavis/conf.d/50-user and add the line $pax='pax'; in the middle:

vi /etc/amavis/conf.d/50-user

use strict;

#
# Place your configuration directives here.  They will override those in
# earlier files.
#
# See /usr/share/doc/amavisd-new/ for documentation and examples of
# the directives you can use in this file
#
$pax='pax';

#------------ Do not modify anything below this line -------------
1;  # ensure a defined return

Afterwards, run these commands to add the clamav user to the amavis group and to restart amavisd-new and ClamAV:

adduser clamav amavis
/etc/init.d/amavis restart
/etc/init.d/clamav-freshclam restart
/etc/init.d/clamav-daemon restart

Now we have to configure Postfix to pipe incoming email through amavisd-new:

postconf -e 'content_filter = amavis:[127.0.0.1]:10024'
postconf -e 'receive_override_options = no_address_mappings'

Afterwards append the following lines to /etc/postfix/master.cf:

vi /etc/postfix/master.cf

[...]
amavis unix - - - - 2 smtp
        -o smtp_data_done_timeout=1200
        -o smtp_send_xforward_command=yes

127.0.0.1:10025 inet n - - - - smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
        -o strict_rfc821_envelopes=yes
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks

Then restart Postfix:

/etc/init.d/postfix restart

Now run

netstat -tap

and you should see Postfix (master) listening on port 25 (smtp) and 10025, and amavisd-new on port 10024:

root@server1:/etc/courier# netstat -tap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 localhost.localdo:10025 *:*                     LISTEN      25911/master
tcp        0      0 localhost.localdo:mysql *:*                     LISTEN      3895/mysqld
tcp        0      0 *:http                  *:*                     LISTEN      4845/apache2
tcp        0      0 *:ssh                   *:*                     LISTEN      649/sshd
tcp        0      0 *:smtp                  *:*                     LISTEN      25911/master
tcp        0      0 localhost.localdo:10024 *:*                     LISTEN      24534/amavisd (mast
tcp        0     52 server1.example.com:ssh 192.168.0.199:4745      ESTABLISHED 847/0
tcp6       0      0 [::]:pop3               [::]:*                  LISTEN      20989/couriertcpd
tcp6       0      0 [::]:imap2              [::]:*                  LISTEN      20921/couriertcpd
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN      649/sshd
tcp6       0      0 [::]:smtp               [::]:*                  LISTEN      25911/master
tcp6       0      0 [::]:imaps              [::]:*                  LISTEN      20958/couriertcpd
tcp6       0      0 [::]:pop3s              [::]:*                  LISTEN      21026/couriertcpd
root@server1:/etc/courier#

 

10 Install Razor, Pyzor And DCC And Configure SpamAssassin

Razor, Pyzor and DCC are spamfilters that use a collaborative filtering network. To install Razor and Pyzor, run

apt-get install razor pyzor

DCC isn't available in the Ubuntu 12.04 repositories, so we install it as follows:

cd /tmp
wget http://www.dcc-servers.net/dcc/source/dcc-dccproc.tar.Z
tar xzvf dcc-dccproc.tar.Z
cd dcc-dccproc-1.3.142
./configure --with-uid=amavis
make
make install
chown -R amavis:amavis /var/dcc
ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd

Now we have to tell SpamAssassin to use these three programs. Edit /etc/spamassassin/local.cf and add the following lines to it:

vi /etc/spamassassin/local.cf

[...]
#dcc
use_dcc 1
dcc_path /usr/local/bin/dccproc

#pyzor
use_pyzor 1
pyzor_path /usr/bin/pyzor

#razor
use_razor2 1
razor_config /etc/razor/razor-agent.conf

#bayes
use_bayes 1
use_bayes_rules 1
bayes_auto_learn 1

Then we must enable the DCC plugin in SpamAssassin. Open /etc/spamassassin/v310.pre and uncomment the loadplugin Mail::SpamAssassin::Plugin::DCC line:

vi /etc/spamassassin/v310.pre

[...]
# DCC - perform DCC message checks.
#
# DCC is disabled here because it is not open source.  See the DCC
# license for more details.
#
loadplugin Mail::SpamAssassin::Plugin::DCC
[...]

You can check your SpamAssassin configuration by executing:

spamassassin --lint

It shouldn't show any errors.

Now there's a little bug in the amavisd-new init script. Open /etc/init.d/amavis...

vi /etc/init.d/amavis

... and comment out the STOP="--stop --quiet --pidfile $PIDFILE --name ${DAEMONNAME}" line and add STOP="--stop --quiet --pidfile $PIDFILE" instead:

[...]
set -e

START="--start --quiet --pidfile $PIDFILE --name ${DAEMONNAME} --startas ${DAEMON}"
#STOP="--stop --quiet --pidfile $PIDFILE --name ${DAEMONNAME}"
STOP="--stop --quiet --pidfile $PIDFILE"
PARAMS=

check_noncompatible_upgrade()
[...]

Restart amavisd-new afterwards:

/etc/init.d/amavis restart

Now we update our SpamAssassin rulesets as follows:

sa-update --no-gpg

We create a cron job so that the rulesets will be updated regularly. Run

crontab -e

to open the cron job editor. Create the following cron job:

23 4 */2 * * /usr/bin/sa-update --no-gpg &> /dev/null

This will update the rulesets every second day at 4.23h.

 

11 Quota Exceedance Notifications

If you want to get notifications about all the email accounts that are over quota, then create the file /usr/local/sbin/quota_notify:

cd /usr/local/sbin/
vi quota_notify

#!/usr/bin/perl -w

# Author <jps@tntmax.com>
#
# This script assumes that virtual_mailbox_base in defined
# in postfix's main.cf file. This directory is assumed to contain
# directories which themselves contain your virtual user's maildirs.
# For example:
#
# -----------/
#            |
#            |
#    home/vmail/domains/
#        |          |
#        |          |
#  example.com/  foo.com/
#                   |
#                   |
#           -----------------
#           |       |       |
#           |       |       |
#         user1/   user2/  user3/
#                           |
#                           |
#                        maildirsize
#

use strict;

my $POSTFIX_CF = "/etc/postfix/main.cf";
my $MAILPROG = "/usr/sbin/sendmail -t";
my $WARNPERCENT = 80;
my @POSTMASTERS = ('postmaster@domain.tld');
my $CONAME = 'My Company';
my $COADDR = 'postmaster@domain.tld';
my $SUADDR = 'postmaster@domain.tld';
my $MAIL_REPORT = 1;
my $MAIL_WARNING = 1;

#get virtual mailbox base from postfix config
open(PCF, "< $POSTFIX_CF") or die $!;
my $mboxBase;
while (<PCF>) {
   next unless /virtual_mailbox_base\s*=\s*(.*)\s*/;
   $mboxBase = $1;
}
close(PCF);

#assume one level of subdirectories for domain names
my @domains;
opendir(DIR, $mboxBase) or die $!;
while (defined(my $name = readdir(DIR))) {
   next if $name =~ /^\.\.?$/;        #skip '.' and '..'
   next unless (-d "$mboxBase/$name");
   push(@domains, $name);
}
closedir(DIR);
#iterate through domains for username/maildirsize files
my @users;
chdir($mboxBase);
foreach my $domain (@domains) {
        opendir(DIR, $domain) or die $!;
        while (defined(my $name = readdir(DIR))) {
           next if $name =~ /^\.\.?$/;        #skip '.' and '..'
           next unless (-d "$domain/$name");
      push(@users, {"$name\@$domain" => "$mboxBase/$domain/$name"});
        }
}
closedir(DIR);

#get user quotas and percent used
my (%lusers, $report);
foreach my $href (@users) {
   foreach my $user (keys %$href) {
      my $quotafile = "$href->{$user}/maildirsize";
      next unless (-f $quotafile);
      open(QF, "< $quotafile") or die $!;
      my ($firstln, $quota, $used);
      while (<QF>) {
         my $line = $_;
              if (! $firstln) {
                 $firstln = 1;
                 die "Error: corrupt quotafile $quotafile"
                    unless ($line =~ /^(\d+)S/);
                 $quota = $1;
            last if (! $quota);
            next;
         }
         die "Error: corrupt quotafile $quotafile"
            unless ($line =~ /\s*(-?\d+)/);
         $used += $1;
      }
      close(QF);
      next if (! $used);
      my $percent = int($used / $quota * 100);
      $lusers{$user} = $percent unless not $percent;
   }
}

#send a report to the postmasters
if ($MAIL_REPORT) {
   open(MAIL, "| $MAILPROG");
   select(MAIL);
   map {print "To: $_\n"} @POSTMASTERS;
   print "From: $COADDR\n";
   print "Subject: Daily Quota Report.\n";
   print "DAILY QUOTA REPORT:\n\n";
   print "----------------------------------------------\n";
   print "| % USAGE |            ACCOUNT NAME          |\n";
   print "----------------------------------------------\n";
   foreach my $luser ( sort { $lusers{$b} <=> $lusers{$a} } keys %lusers ) {
      printf("|   %3d   | %32s |\n", $lusers{$luser}, $luser);
      print "---------------------------------------------\n";
   }
        print "\n--\n";
        print "$CONAME\n";
        close(MAIL);
}

#email a warning to people over quota
if ($MAIL_WARNING) {
        foreach my $luser (keys (%lusers)) {
           next unless $lusers{$luser} >= $WARNPERCENT;       # skip those under quota
           open(MAIL, "| $MAILPROG");
           select(MAIL);
           print "To: $luser\n";
      map {print "BCC: $_\n"} @POSTMASTERS;
           print "From: $SUADDR\n";
           print "Subject: WARNING: Your mailbox is $lusers{$luser}% full.\n";
           print "Reply-to: $SUADDR\n";
           print "Your mailbox: $luser is $lusers{$luser}% full.\n\n";
           print "Once your e-mail box has exceeded your monthly storage quota\n";
      print "your monthly billing will be automatically adjusted.\n";
      print "Please consider deleting e-mail and emptying your trash folder to clear some space.\n\n";
           print "Contact <$SUADDR> for further assistance.\n\n";
           print "Thank You.\n\n";
           print "--\n";
           print "$CONAME\n";
           close(MAIL);
        }
}

Make sure that you adjust the variables at the top (especially the postmaster@domain.tld email address).

We must make the file executable:

chmod 755 quota_notify

Run

crontab -e

to create a cron job for that script:

0 0 * * * /usr/local/sbin/quota_notify &> /dev/null
Share this page:

26 Comment(s)

Add comment

Comments

From: at: 2012-05-17 10:58:27

Hi....

make: *** [maildir.o] Error 1
make: Leaving directory `/usr/src/postfix-2.9.1/src/smtpd'
make[1]: *** [update] Error 1
make[1]: Leaving directory `/usr/src/postfix-2.9.1'
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

 help what to do?

From: at: 2012-10-23 19:27:15

I had the same problem with Ubuntu 12.04 and postfix 2.9.3 with vda-patch 2.9.1. German language breaks postfix compilation:

../../lib/libdns.a: undefined reference to `__res_search'
../../lib/libdns.a: undefined reference to `__dn_expand'
collect2: ld gab 1 als Ende-Status zurück
make: *** [smtpd] Fehler 1</pre>

The reason of this error: missing -lresolv
I found a problem in Postfix makedefs file.

gcc -print-search-dirs | sed -n '/^libraries: =/s/libraries: =//p' didn't work for me, because i have Ubuntu in german language.
The gcc -print-search-dirs output is "Bibliotheken", german word of "libraries".

After changing the sed command the SYSLIBS variable filled in correctly and Postfix compiles fine.

http://www.howtoforge.com/forums/showthread.php?p=287215

From: Anonymous at: 2012-05-22 15:01:18

If you got that error above (format not a string literal and no format arguments), it's better to change the code like this:

 dsb_simple(why, "5.2.2", limit_message) -> dsb_simple(why, (char*)"5.2.2", "%s", limit_message);

 or

 dsb_simple(why, "2.0.0", "delivers to maildir") -> dsb_simple(why, (char*)"2.0.0", (char*)"delivers to maildir");

 

I think it's more elegant than disable hardening build. :)

From: at: 2012-06-20 07:01:43

One more step is necessary to get phpMyAdmin running.

Either:

cat >> /etc/apache2/apache2.conf <<'EOF'

# Include the phpadmin config file
Include /etc/phpmyadmin/apache.conf
EOF

or

sudo ln -s /etc/phpmyadmin/apache.conf /etc/apache2/conf.d/phpmyadmin.conf

After either of those (not both!), reload apache2:

/etc/init.d/apache2 reload

From: tcarrondo at: 2012-06-28 14:09:38

Before 'dpkg-buildpackage' don't we need to 'debchange -i' so that the compiled postfix package isn't reverted in next 'apt-get upgrade'?

From: Anonymous at: 2013-01-28 19:53:04

Great tutorial Falko! Thank you!

From: Anonymous at: 2013-03-08 19:57:42

Recent 12.04 brings postfix 2.9.6 at this time there is no quota patch for 2.9.6 available, but the patch for 2.9.5 can be applied.

From: Anonymous at: 2013-03-09 09:44:52

update: patching and compiling works, but it seems to brake something as the chroot is being disabled for postfix and even if re-enabled it outputs errors like postfix/qmgr[1547]: warning: private/smtp socket: malformed response so i reverted back to the unpatched postfix.

From: Felipe Alcacibar at: 2014-03-11 04:44:01

To avoid the upgrade of the postfix package you can use the following command:

 # echo "postfix hold" | dpkg --set-selections

From: iainH at: 2012-05-24 12:37:49

Falko,

thanks so much for a clear, concise and working tutorial. This has saved me so much experimentation - the quota and amavis-wrapped services in particular represent a lot of experience that I gratefully re-use.

 The only extra thing that I had to get my system working after the tutorial (postfix and courier-imapwas already up and running) was to open these additional firewall ports as follows:

Submission SMTP:
TCP 587
(Shorewall "Mail" macro includes ports 25, 465, 587)
DCC Server:
UDP 6277
Pyzor's rules sets Server
UDP 24441
Razor2's access to Cloudmark.com's rules sets server
UDP 2703

From: Ed at: 2012-07-11 13:50:00

Shouldn't

postconf -e 'virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, mysql:/etc/postfix/mysql-virtual_email2email.cf'

be

postconf -e 'virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email.cf'

From: Ed at: 2012-07-18 10:47:47

It looks like there might be a bug with gamin that is required by courier-imap. You lose imap connection to the server at random times after a reboot. 

This is also related when having a web mail client like Round Cube or Squirrel Mail installed.

I found a solution at http://ubuntuforums.org/showthread.php?s=fba71f803c489876a75fb08df822a930&p=11856138 . Running stable now. 

 

 

 

From: Anonymous at: 2012-08-04 15:35:21

Great guide !!! It take sometime to find out that I forgot to change /etc/default/saslauthd : START=yes, my bad :)

From: Antonio Delgado at: 2013-07-04 18:22:17

I found out (thanks to http://etbe.coker.com.au/2012/06/20/sasl-authentication-wheezy/ ) that using this manual with ubuntu and postfix 2.9.6-1~12.04 sasl2-bin 2.1.25.dfsg1-3ubuntu the file /etc/postfix/sasl/smtpd.conf need to be changed to:

pwcheck_method: saslauthd
mech_list: plain login
allow_plaintext: true
auxprop_plugin: sql
sql_engine: mysql
sql_hostnames: 127.0.0.1
sql_user: mail_admin
sql_passwd: mail_admin_password
sql_database: mail
sql_select: select password from users where email = '%u@%r'

(using the right database connection information)

From: Anonymous at: 2014-04-20 18:28:43

What is the difference, since I can't find any?

From: Anonymous at: 2014-04-12 12:31:34

I note there are many files in this tut with this string:

 mail_admin_password

Am I suppose to be inserting this literal string in files being created/edited or am I suppose to be injecting the actual mail_admin's password?

From: Anonymous at: 2012-05-21 09:34:53

when I tried to login by using user name and password I got this message

ERROR: ERROR: Connection dropped by IMAP server.

From: Anonymous at: 2012-07-06 10:08:49

Yep same error !

From: RojasIT at: 2012-08-03 19:19:37

Try to send it a email at your mail account

From: Armando at: 2013-07-08 18:43:07

I have an err, I just can sent an emails but I can't receive emails, somebody can help me please..

From: Anonymous at: 2012-08-02 19:39:38

I had the same error

From: Anonymous at: 2012-08-21 15:33:48

You should just create your Maildir directory in your home folder

From: Aristotle Jones at: 2012-08-31 14:35:58

I went through the Debian version of this tutorial and got the same, I then went through this one and it worked fine.  During my troubleshooting of my debian setup I discovered two things, one was a typo in the squirrelmail plugin config, I suggest copy and past of the entire file where ever possible in this tutorial.  Also, my mailbox wasn't properly created when I did the mailx command.  I'm not sure why it didn't work, but I suspect it was due to my incorrect setup of my root alias in previous postfix steps.  Not sure if this will help you out or not, but one error will cause havoc in this tutorial.

From: Anonymous at: 2012-10-23 20:03:02

I got the same problem even if I sent an e-mail to the new mailbox. I solved it by editing /etc/postfix/main.cf.

Replace :

mydestination = domain.tld, localhost, localhost.localdomain

By :

 mydestination = localhost, localhost.localdomain

And restart postfix :

 /etc/init.d/postfix restart

Finally send an e-mail to the created mailbox and enjoy !

From: Marlon Mann at: 2012-12-20 00:16:51

After hours if not days trying to wade through frankly not very clear tutorials on setting up an email system on Ubuntu, I'm so glad I found this. 

I followed the instructions step by step and, well, it just works.

I'm so very grateful. Keep up the good work!

From: Anonymous at: 2013-09-12 19:09:57

Just saying thanks for a great how-to. Using database entries to manage the mail system has made my life so much easier. I can also confirm that the tutorial still works fine with latest patches on Ubuntu 12.04. I've been running a system based on this configuration all year with no problems.