The Perfect SpamSnake - Ubuntu Jeos 12.04 LTS Precise Pangolin - Page 2

7. Install MailScanner (Apparmor, Clamav, DCC, Pyzor, Razor and Spamassassin)

apt-get install razor pyzor clamav-daemon libclamav6 apparmor -y

Apparmor configuration for Clamav

Add clamav to the www-data group so that it can access the directory:

usermod -a -G www-data clamav

Now edit the profile for clamd:

vi /etc/apparmor.d/usr.sbin.clamd

and add the following to the list of folders:

#clamav
   /var/spool/MailScanner/** rw,
   /var/spool/MailScanner/incoming/** rw,

Reload apparmor:

/etc/init.d/apparmor reload

DCC 32bit/64bit Configuration

Install DCC from .deb source:

cd /tmp
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-common_1.3.130-0ubuntu1~ppa2~quantal1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb && dpkg -i dcc-common_1.3.130-0ubuntu1~ppa2~quantal1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-client_1.3.130-0ubuntu1~ppa2~quantal1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb && dpkg -i dcc-client_1.3.130-0ubuntu1~ppa2~quantal1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb

Test our installation with:

cdcc info

You should get 'requests ok' from the servers.

Pyzor Configuration

Because pyzor doesn’t work with python2.6 very well, the workaround is to append the following to the first line of /usr/bin/pyzor to make it look like:

#!/usr/bin/python -Wignore::DeprecationWarning

Here we supply the IP address of the Pyzor server to Pyzor. This will create the server's IP address in a servers file therein. Then it will test the connection. If you are behind a firewall, open port 24441/udp in and out to your server. While you're at it also open up 6277/udp for DCC, 2703/tcp for Razor and 783/tcp for SpamAssassin:

mkdir /var/lib/MailScanner
pyzor --homedir=/var/lib/MailScanner discover
pyzor ping

Razor Configuration

Create the .razor configuration:

cd && rm /etc/razor/razor-agent.conf
mkdir /var/lib/MailScanner/.razor
razor-admin -home=/var/lib/MailScanner/.razor -create
razor-admin -home=/var/lib/MailScanner/.razor -discover
razor-admin -home=/var/lib/MailScanner/.razor -register 

vi /var/lib/MailScanner/.razor/razor-agent.conf

and add/update the follwing

debuglevel      = 0
razorhome 	= /var/lib/MailScanner/.razor/

Install dependencies:

apt-get install libconvert-tnef-perl libdbd-sqlite3-perl libfilesys-df-perl libmailtools-perl libmime-tools-perl libmime-perl libnet-cidr-perl libsys-syslog-perl libio-stringy-perl libfile-temp-perl libole-storage-lite-perl libarchive-zip-perl libsys-hostname-long-perl libnet-cidr-lite-perl libhtml-parser-perl libdb-file-lock-perl libnet-dns-perl libncurses5-dev libdigest-hmac-perl libnet-ip-perl liburi-perl libfile-spec-perl spamassassin libnet-ident-perl libmail-spf-perl libmail-dkim-perl dnsutils libio-socket-ssl-perl gdebi-core -y
wget http http://launchpadlibrarian.net/85191561/libdigest-sha1-perl_2.13-2build2_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb && dpkg -i libdigest-sha1-perl_2.13-2build2_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb

Download and install the latest MailScanner:

cd /usr/src && wget http://mailscanner.info/files/4/tar/MailScanner-install-4.84.5-3.tar.gz
tar xvfz MailScanner-install-4.84.5-3.tar.gz && cd MailScanner-install-4.84.5
./install.sh

Add the following to cron:

37      5 * * *  /opt/MailScanner/bin/update_phishing_sites &> /dev/null
07      * * * *  /opt/MailScanner/bin/update_bad_phishing_sites &> /dev/null
58     23 * * * /opt/MailScanner/bin/clean.quarantine &> /dev/null
42      * * * *  /opt/MailScanner/bin/update_virus_scanners &> /dev/null
3,23,43 * * * *  /opt/MailScanner/bin/check_mailscanner &> /dev/null

Fix Clamav autoupdate

vi /opt/MailScanner/etc/virus.scanners.conf

clamav          /opt/MailScanner/lib/clamav-wrapper     /usr/

SpamAssassin

First we need to disable the default SpamAssassin configuration file:

mv /etc/spamassassin/local.cf /etc/spamassassin/local.cf.disabled

Now let's backup the SpamAssassin configuration file in MailScanner then edit:

cp /opt/MailScanner/etc/spam.assassin.prefs.conf /opt/MailScanner/etc/spam.assassin.prefs.conf.back

SpamAssassin SQL Bayes

Pre-requisities: You'll need the perl-DBI and perl-DBD-MySQL modules installed.

Assumptions and Variables:

SpamAssassin Bayes Database Name: sa_bayes
SpamAssassin Bayes Database UserName: sa_user
SpamAssassin Bayes Database Password: sa_password

Create the MySQL database on the server where you intend on storing the bayesian information.

mysql -u root -p
mysql> create database sa_bayes;
mysql> GRANT ALL ON sa_bayes.* TO [email protected] IDENTIFIED BY 'sa_password';
mysql> flush privileges;

Import database structure:

mysql -u sa_user -p sa_bayes < /usr/share/doc/spamassassin/sql/bayes_mysql.sql

*Note: If errors about MyISAM appears, edit bayes_mysql.sql and change TYPE to ENGINE.

vi /etc/spamassassin/v310.pre

and enable DCC:

loadplugin  Mail::SpamAssassin::Plugin::DCC

Create the following to prevent an error in a lint test:

mkdir /var/www/.spamassassin

vi /opt/MailScanner/etc/spam.assassin.prefs.conf

and add the following to the top:

#pyzor
use_pyzor 1
pyzor_options --homedir /var/lib/MailScanner/
   
#razor
use_razor2 1
razor_config /var/lib/MailScanner/.razor/razor-agent.conf

Fix DCC path:

dcc_path /usr/bin/dccproc

Update header string:

bayes_ignore_header X-YOURDOMAIN-COM-MailScanner
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamCheck
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamScore
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-Information
#use_auto_whitelist 0

"YOURDOMAIN-COM" should be replaced with whatever you used for "%org-name%" in the MailScanner.conf file. Leave the "X-" in place. This is the same orgname used in the MailScanner.conf above.

Add sql connection string to bottom:

bayes_store_module Mail::SpamAssassin::BayesStore::SQL
bayes_sql_dsn DBI:mysql:sa_bayes:localhost
bayes_sql_username sa_user
bayes_sql_password sa_password
bayes_sql_override_username root

Add it to cron:

30 01 * * * /usr/bin/sa-learn --force-expire --sync -p /opt/MailScanner/etc/spam.assassin.prefs.conf 

Install missing perl packages:

perl -MCPAN -e shell
install IP::Country::Fast
install Encode::Detect
install Crypt::OpenSSL::RSA

Set permissions to bring it all together:

chown -R postfix:www-data /var/spool/postfix/hold
chmod -R ug+rwx /var/spool/postfix/hold 

Test out the setup:

spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf --lint

Check for lines like:

debug: bayes: Database connection established
debug: bayes: found bayes db version 3
debug: bayes: Using userid: 2

You should see lines come up with DCC, Pyzor and Razor that say loading plugin and hopefully no errors.

MailScanner Configuration

We need to make a directory for SpamAssassin in the spool and give postfix permissions to it, if you run sa-learn --force as root, bayes databese that is stored in these directories will change to root:root and spamassassin will error looking at the db. Just keep an eye on the mail.log and you'll remember to change the permissions back. Also disable the MailScanner default configs:

mkdir /var/spool/MailScanner/spamassassin

Backup your MailScanner.conf file:

cp /opt/MailScanner/etc/MailScanner.conf /opt/MailScanner/etc/MailScanner.conf.dist

vi /usr/src/mailscanner.sh

with the following content:

*Note: This step populates the MailScanner.conf file.

sed -i "/^%org-name% =/ c\%org-name% =orgname" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^%org-long-name% =/ c\%org-long-name% = longorgname" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^%web-site% =/ c\%web-site% = www.domain.tld" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Run As User =/ c\Run As User = postfix" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Run As Group =/ c\Run As Group = www-data" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Incoming Work Group =/ c\Incoming Work Group = clamav" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Incoming Work Permissions =/ c\Incoming Work Permissions = 0640" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Incoming Queue Dir =/ c\Incoming Queue Dir = /var/spool/postfix/hold" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Outgoing Queue Dir =/ c\Outgoing Queue Dir = /var/spool/postfix/incoming" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^MTA =/ c\MTA = postfix" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Quarantine User =/ c\Quarantine User = root" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Quarantine Group =/ c\Quarantine Group = www-data" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Quarantine Permissions =/ c\Quarantine Permissions = 0660" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Quarantine Whole Message =/ c\Quarantine Whole Message = yes" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Virus Scanners =/ c\Virus Scanners = clamd" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Monitors for ClamAV Updates =/ c\Monitors for ClamAV Updates = /var/lib/clamav/*.cld /var/lib/clamav/*.cvd" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Clamd Socket =/ c\Clamd Socket = /var/run/clamav/clamd.ctl" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Clamd Lock File =/ c\Clamd Lock File = /var/run/clamav/clamd.pid" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Spam Subject Text =/ c\Spam Subject Text = ***SPAM***" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Spam Actions =/ c\Spam Actions = deliver store" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^High Scoring Spam Actions =/ c\High Scoring Spam Actions = store delete" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^Non Spam Actions =/ c\Non Spam Actions = deliver store" /opt/MailScanner/etc/MailScanner.conf
sed -i "/^SpamAssassin User State Dir =/ c\SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin" /opt/MailScanner/etc/MailScanner.conf

*Note: Make sure to change the items in red before running the script.

Make it executable and run it:

chmod +x mailscanner.sh
./usr/src/mailscanner.sh

MailScanner Startup Script:

vi /etc/init.d/mailscanner

with the following content:

#! /bin/sh
   ### BEGIN INIT INFO
   # Provides:          MailScanner daemon
   # Required-Start:    $local_fs $remote_fs
   # Required-Stop:     $local_fs $remote_fs
   # Default-Start:     2 3 4 5
   # Default-Stop:      0 1 6
   # Short-Description: Controls mailscanner instances
   # Description:       MailScanner is a queue-based spam/virus filter
   ### END INIT INFO
   # Author: Simon Walter <[email protected]>
   # PATH should only include /usr/* if it runs after the mountnfs.sh script
   PATH=/usr/sbin:/usr/bin:/bin:/sbin:/opt/MailScanner/bin
   DESC="mail spam/virus scanner"
   NAME=MailScanner
   PNAME=mailscanner
   DAEMON=/opt/MailScanner/bin/$NAME
   STARTAS=MailScanner
   SCRIPTNAME=/etc/init.d/$PNAME
   CONFFILE=/opt/MailScanner/etc/MailScanner.conf
   # Exit if the package is not installed
   [ -x "$DAEMON" ] || exit 0
   run_nice=0
   stopped_lockfile=/var/lock/subsys/MailScanner.off
   # Read configuration variable file if it is present
   [ -r /etc/default/$PNAME ] && . /etc/default/$PNAME
   # Load the VERBOSE setting and other rcS variables
   . /lib/init/vars.sh
   # Define LSB log_* functions.
   # Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
   . /lib/lsb/init-functions
   # sanity check for permissions
   fail()
   {
   echo >&2 "$0: $1"
   exit 1
   }
   check_dir()
   {
   if [ ! -d $1 ]; then
   mkdir -p "$1" || \
   fail "directory $1: does not exist and cannot be created"
   fi
   actual="$(stat -c %U $1)"
   if [ "$actual" != "$2" ]; then
   chown -R "$2" "$1" || \
   fail "directory $1: wrong owner (expected $2 but is $actual)"
   fi
   actual="$(stat -c %G $1)"
   if [ "$actual" != "$3" ]; then
   chgrp -R "$3" "$1" || \
   fail "directory $1: wrong group (expected $3 but is $actual)"
   fi
   }
   user=$(echo $(awk -F= '/^Run As User/ {print $2; exit}' $CONFFILE))
   group=$(echo $(awk -F= '/^Run As Group/ {print $2; exit}' $CONFFILE))
   check_dir /var/spool/MailScanner       ${user:-postfix} ${group:-www-data}
   check_dir /var/lib/MailScanner         ${user:-postfix} ${group:-www-data}
   check_dir /var/run/MailScanner         ${user:-postfix} ${group:-www-data}
   check_dir /var/lock/subsys	${user:-root}	${group:-root} #Required to Create Folder
   check_dir /var/lock/subsys/MailScanner ${user:-postfix} ${group:-www-data}
   #
   # Function that starts the daemon/service
   #
   do_start()
   {
   # Return
   #   0 if daemon has been started
   #   1 if daemon was already running
   #   2 if daemon could not be started
   start-stop-daemon --start --quiet --startas $STARTAS --name $NAME --test > /dev/null \
   || return 1
   start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix:www-data --exec $DAEMON --name $NAME -- $DAEMON_ARGS \
   || return 2
   # Add code here, if necessary, that waits for the process to be ready
   # to handle requests from services started subsequently which depend
   # on this one.  As a last resort, sleep for some time.
   # Set lockfile to inform cronjobs about the running daemon
   RETVAL="$?"
   if [ $RETVAL -eq 0 ]; then
   touch /var/lock/subsys/mailscanner
   rm -f $stopped_lockfile
   fi
   if [ $RETVAL -eq 0 ]; then
   echo "MailScanner Started"
   fi
   }
   #
   # Function that stops the daemon/service
   #
   do_stop()
   {
   # Return
   #   0 if daemon has been stopped
   #   1 if daemon was already stopped
   #   2 if daemon could not be stopped
   #   other if a failure occurred
   start-stop-daemon --stop --retry=TERM/30 --name $NAME
   RETVAL="$?"
   [ "$RETVAL" = 2 ] && return 2
   # Remove lockfile for cronjobs
   if [ $RETVAL -eq 0 ]; then
   rm -f /var/lock/subsys/mailscanner
   touch $stopped_lockfile
   fi
   if [ $RETVAL -eq 0 ]; then
   echo "MailScanner Stopped"
   fi
   }
   #
   # Function that sends a SIGHUP to the daemon/service
   #
   do_reload() {
   start-stop-daemon --stop --signal 1 --quiet --name $NAME
   return 0
   }
   case "$1" in
   start)
   [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
   do_start
   case "$?" in
   0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
   2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
   esac
   ;;
   stop)
   [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
   do_stop
   case "$?" in
   0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
   2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
   esac
   ;;
   restart|force-reload)
   #
   # If the "reload" option is implemented then remove the
   # 'force-reload' alias
   #
   log_daemon_msg "Restarting $DESC" "$NAME"
   do_stop
   case "$?" in
   0|1)
   do_start
   case "$?" in
   0) log_end_msg 0 ;;
   1) log_end_msg 1 ;; # Old process is still running
   *) log_end_msg 1 ;; # Failed to start
   esac
   ;;
   *)
   # Failed to stop
   log_end_msg 1
   ;;
   esac
   ;;
   *)
   echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
   exit 3
   ;;
   esac
 exit 0

Make it executable:

chmod +x /etc/init.d/mailscanner

Create Symlinks for mailscanner script to work:

chmod 755 /etc/init.d/mailscanner
update-rc.d mailscanner defaults
ln -s /opt/MailScanner/bin/Quick.Peek /usr/sbin/Quick.Peek

Start the system:

/etc/init.d/mailscanner start
/etc/init.d/postfix start

Check your logs for errors:

tail -f /var/log/mail.log

Check your mail.log (tail –f /var/log/mail.log) and you should see the following:

Jun 13 12:18:23 hoshi MailScanner[26388]: MailScanner E-Mail Virus Scanner version 4.84.5 starting...

Congratulations - you now have MailScanner logging to MySQL.