Postfix Spam Filter using Ubuntu Dapper, MailScanner, SpamAssassin, Razor, Pyzor, DCC and ClamAV - Page 4

Want to support HowtoForge? Become a subscriber!
 
Submitted by fdalmoro (Contact Author) (Forums) on Tue, 2006-07-11 15:31. ::

3 Pyzor, Razor, DCC, SpamAssassin and MailScanner Configuration

3.1 Pyzor Configuration

We need to change some permissions on pyzor first:

chmod -R a+rX /usr/share/doc/pyzor /usr/bin/pyzor /usr/bin/pyzord


This next command will have to be modified if you have a different version of python. Try locate pyzor.

chmod -R a+rX /usr/lib/site-python/pyzor


Here we supply the IP address of the Pyzor server to Pyzor. This will create a .pyzor directory in both user's home directories, and place the server's IP address in a servers file therein. Then it will test the connection. If you are behind a firewall, open port 24441/udp in and out to your server. While you're at it also open up 6277/udp for DCC, 2703/tcp for Razor and 783/tcp for SpamAssassin:

pyzor ping


Usually you'll get a timeout from the pyzor ping so don't worry about it. We'll test again later.

If in the future the IP address of the server changes, you will need to run through this section again. You can find the address of the current Pyzor server here http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x.

If Pyzor is working, you will see "Pyzor: got response:" Pyzor queries a Pyzor server in much the same way your computer queries a DNS server. The only practical difference is the port number that is used. If Pyzor is not working, you might need to open up the port on your firewall or the Pyzor server is busy.

3.2 Razor Configuration

Create a default .razor configuration under root home directory:

cd
rm /etc/razor/razor-agent.conf
razor-admin -create
razor-admin -create


Razor v2 requires reporters to be registered so their reputations can be computed over time and they can participate in the revocation mechanism. Registration is done with razor-admin -register. When razor-admin -register is invoked as root, it negotiates a registration with the Nomination Server and writes the identity information in /root/.razor/identity-username. Manually invoked it in one of the following ways:

1. To register user:foo and password:s1kret (foo and s1kret are examples) :

razor-admin -register -user=foo -pass=s1kr3t


2. To register with an email address and have the password assigned:

razor-admin -register -user=foo@bar.com


3. To have both (random) username and password assigned:

razor-admin -register


I usually just do number 3. Make the following changes to /root/.razor/razor-agent.conf:

vi /root/.razor/razor-agent.conf


Change debuglevel = 3 to debuglevel = 0 (yes zero not "o"). This will prevent Razor from filling up your drive with debug information. Also we will move these configs someplace that the Postfix user can read them so add the razorhome line to the end of the file. Those two lines should look like this when done:

debuglevel             = 0
razorhome             = /var/lib/MailScanner/.razor/

We will test Razor later. man razor-agent.conf or go to http://razor.sourceforge.net/docs/razor-agent.conf.php for more information on Razor.

3.3 DCC Setup and Configuration

Install DCC:

apt-get install dcc-client


We are not running a DCC server, so we don't need to waste time checking ourselves: If you are a large organization (100,000 messages per day), you should investigate running your own server.

Once the installation is done run:

cdcc "delete 127.0.0.1"
cdcc "delete 127.0.0.1 Greylist"


Test our installation with:

cdcc info


You should get 'requests ok' from the servers.

4 MailScanner, ClamAV and SpamAssassin Configuration

4.1 MailScanner and ClamAV

Stop Postfix:

postfix stop


Install the packages:

apt-get install mailscanner clamav


Update ClamAV virus defenitions:

freshclam


Let's start with MailScanner. The MailScanner that was just installed from the repositories is a very old version so we will now remove it and install the MailScanner package from source

Download the tarball from http://www.mailscanner.info/downloads.html ... At the time of this writing it is at version 4.56.8-1 and the tarball link is http://www.mailscanner.info/files/4/tar/MailScanner-install-4.56.8-1.tar.gz. Then install MailScanner using the install.sh script.

cd
apt-get remove mailscanner
wget http://www.mailscanner.info/files/4/tar/MailScanner-install-4.56.8-1.tar.gz
tar zxvf MailScanner-install-4.56.8-1.tar.gz
cd MailScanner-install-4.56.8
./install.sh


Ignore the message about the cron lines that we need to add to cron for now.

Once that is done, we need to make a directory for SpamAssassin in the spool and give postfix permissions to it, if you run sa-learn --force as root, bayes databese that is stored in these directories will change to root:root and spamassassin will error looking at the db. Just keep an eye on the mail.log and you'll remember to change the permissions back. Also disable the MailScanner default configs:

mkdir /var/spool/MailScanner/spamassassin
mv /etc/MailScanner /etc/MailScanner.dist


Backup your MailScanner.conf file:

cp /opt/MailScanner/etc/MailScanner.conf /opt/MailScanner/etc/MailScanner.conf.back


Edit MailScanner.conf:

vi /opt/MailScanner/etc/MailScanner.conf


Change the following parameters in MailScanner.conf:

%org-name% = YOURDOMAIN-COM
%org-long-name% = Your Company Long Name INC
%web-site% = www.yourdomain.com
Run As User = postfix
Run As Group = postfix
Incoming Queue Dir = /var/spool/postfix/hold
Outgoing Queue Dir = /var/spool/postfix/incoming
MTA = postfix
Virus Scanners = clamav
Spam Subject Text = [SPAM]
Send Notices = no
Spam List = ORDB-RBL SBL+XBL
Required SpamAssassin Score = 6
High SpamAssassin Score = 10
Spam Actions = deliver striphtml
High Scoring Spam Actions = delete
Rebuild Bayes Every = 86400
Wait During Bayes Rebuild = yes
SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin

The first 9 lines are basically required in order for everything to work, the rest are recommended. The MailScanner.conf is well documented so please read the notes there if you have any questions about the rest of the options we changed. Poke around this file from top to bottom.

Also take a look at the section "Removing/Logging dangerous or potentially offensive content" in the MailScanner.conf file. I had to disable most of these because clients were complaining about '{Disarmed}' messages.

4.2 SpamAssassin

First we need to disable the default SpamAssassin configuration file:

mv /etc/spamassassin/local.cf /etc/spamassassin/local.cf.disabled


Now lets backup the SpamAssassin configuration file in MailScanner then edit:

cp /opt/MailScanner/etc/spam.assassin.prefs.conf /opt/MailScanner/etc/spam.assassin.prefs.conf.back


vi /opt/MailScanner/etc/spam.assassin.prefs.conf


Add these two lines to the top of spam.assassin.prefs.conf:

pyzor_options --homedir /var/lib/MailScanner/
razor_config /var/lib/MailScanner/.razor/razor-agent.conf

Change where SpamAssassing looks for the Bayes database, comment out the default bayes_path or change it accordingly:

#bayes_path /var/lib/MailScanner/bayes
bayes_path /var/spool/MailScanner/spamassassin/bayes

Look for these lines and change them accordingly:

bayes_ignore_header X-YOURDOMAIN-COM-MailScanner
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamCheck
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamScore
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-Information

"YOURDOMAIN-COM" should be replaced with whatever you used for "%org-name%" in the MailScanner.conf file. Leave the "X-" in place.

Make sure that "bayes_auto_expire 0" is not commented out:

bayes_auto_expire 0

Edit the SpamAssassin v310.pre to enable Razor and DCC

vi /etc/spamassassin/v310.pre


Uncomment the following lines:

loadplugin Mail::SpamAssassin::Plugin::DCC
loadplugin Mail::SpamAssassin::Plugin::Razor2

5 Bring it all Together

Copy over the Pyzor and Razor configs to someplace that the Postfix user will be able to read them:

cp -R /root/.pyzor /var/lib/MailScanner
cp -R /root/.razor /var/lib/MailScanner


Now that we have everything in there, set the correct permissions:

chown -R postfix.postfix /var/spool/MailScanner/
chown -R postfix.postfix /var/lib/MailScanner/


Let's see if SpamAssassin is happy:

su postfix -p -c 'spamassassin -x -D -C /opt/MailScanner/etc/spam.assassin.prefs.conf --lint'


You should see lines come up with DCC, Pyzor and Razor that say loading plugin and hopefully no errors.

NOTE: If your ever run the sa-learn, remember to run it like this, su postfix -p -c 'sa-learn --sync --force-expire -C /opt/MailScanner/etc/spam.assassin.prefs.conf' otherwise when SpamAssassin rebuilds the bayes databese it will not be able to read it.

If everything is looking dandy, continue, if not, troubleshoot and then continue.

Finishing up this part we need to add cron jobs that will clean/update/run Mailscanner, you probably saw the message about this after the MailScanner install script finished. The reason we do it now is because we don't want MailScanner starting while we finish the SpamAssassin configuration.

crontab -e


Add these lines:

37      5 * * * /opt/MailScanner/bin/update_phishing_sites
58     23 * * * /opt/MailScanner/bin/clean.quarantine
42      * * * * /opt/MailScanner/bin/update_virus_scanners
3,23,43 * * * * /opt/MailScanner/bin/check_mailscanner

We need to add a line to rc.local so that MailScanner starts on a reboot:

vi /etc/rc.local


Before the "Exit 0" line add:

/opt/MailScanner/bin/check_mailscanner

Might as well link the "check_mailscanner" file into the bin directory. This way you can run it whenever you need to restart MailScanner:

cd /usr/bin
ln -s /opt/MailScanner/bin/check_mailscanner check_mailscanner


Just to give you a clean log to look at and reboot:

rm /var/log/mail.log
reboot


If you see some errors on reboot when starting MailScanner about the Perl Sys/Hostname/Long.pm you need to install it like so:

perl -MCPAN -e shell


If it asks you to configure it now say no, unless you know what you're doing. Now run the following to install the module:

install Sys::Hostname::Long

When that's done type "quit" to go back to the console. Reboot or start MailScanner with the "check_mailscanner" script.

At this point you should have a fully functional spamfilter. Take a look at tail -f /var/log/mail.log, it should be pretty much clear of errors.

This README should come in handy for future use. Copy it to your spamfilter for a quick reference.

POSTFIX, UBUNTU, MAILSCANNER README
**************************************************
ADD DOMAIN
**************************************************
- Edit '/etc/postfix/relay_recipients', 'relay_domains' and 'transport'.
- Run 'postmap /etc/postfix/relay_recipients'. Same for 'relay_domains' and 'transport' after edit to add domains into db file.
- 'postfix reload' for postfix to read new db files.
- To add users to domains edit the 'relay_recipients' and 'postmap' it.
**************************************************
CONTROLLING BLIST WLIST
**************************************************
- Edit /etc/postfix/sender_access
- Run 'postmap /etc/postfix/sender_access'
- Run 'postfix reload'
**************************************************
OTHER CONFIG FILES
**************************************************
- To edit MailScanner Settings "/opt/MailScanner/etc/MailScanner.conf"
- To edit spamassassin Settings "/opt/MailScanner/etc/spam.assassin.prefs.conf"
- To edit clamav Settings "/etc/clamav/clamd.conf"
**************************************************
MISC
**************************************************
- Run 'LINUX2' if postfix errors appear about files not matching in the jailroot.
- Run 'postfix check' to see if postfix is synched with jailroot.
- Run newaliases to refresh the /etc/postfix/aliases database if any changes are made on that file..
- 'mailq' and 'qshape' to check queue.
- 'check_mailscanner' to restart MailScanner.

Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by bkeshet (registered user) on Sun, 2008-03-16 14:15.

when regisrering with razor you can recieve a
Error 202 while performing register, aborting.
Turns-out you need to run the razor-admin with a -discover first if you come across this error. In general it’s some sort of network error.
razor-admin -discoverrazor-admin -createrazor-admin -register
thanks to http://devnulled.com/content/2005/03/razor-error-202-while-performing-register-aborting/
for the solution