The Perfect SpamSnake - Ubuntu Jaunty Jackalope

Want to support HowtoForge? Become a subscriber!
 
Submitted by Rocky (Contact Author) (Forums) on Wed, 2009-06-10 18:29. :: Anti-Spam/Virus | Ubuntu | Postfix

The Perfect SpamSnake - Ubuntu Jaunty Jackalope

Author: Mohammed Alli

Postfix w/Bayesian Filtering and Anti-Backscatter (Relay Recipients), Apache, Mysql, Dnsmasq, MailScanner (Spamassassin, ClamAV, Pyzor, Razor, DCC-Client), MailWatch, SPF Checks, FuzzyOcr, PDF/XLS/Phishing Sanesecurity Signatures, Postfix-GLD (Greylisting Optional), Logwatch Statistical Reporting (Optional), Outgoing Disclaimer with alterMIME (Optional), FireHOL (Iptables Firewall)

Version 2.5

This tutorial shows how to set up an Ubuntu Jaunty Jackalope based server as a spamfilter in Gateway mode. In the end, you will have a SpamSnake Gateway which will relay clean emails to your MTA. You will also be able to view your incoming queue, train your SpamSnake and carry out a few more advanced operations via MailWatch.

I cannot offer any guarantees that this will work for you, the same way it's working for me.

I will use the following software:

  • Web Server: Apache 2 with PHP 5
  • Database Server: MySQL 5.0
  • Mail Server: Postfix
  • Caching DNS Server: Dnsmasq
  • MailScanner: MailScanner v4.76
  • MailWatch: MailWatch v1.0.4

Credit goes to the guys at HowtoForge and the developers of MailScanner, MailWatch, ClamAV, Apache, Mysql and Postfix.

 

Install the base system using the minimal option.

1. Get root Privileges

Enable the root login by running the following and giving root a password. You can then directly log in as root:

sudo passwd root

 

 2. Install vim-nox (Optional)

I'll use vi as my text editor in this tutorial. The default vi program has some strange behaviour on Ubuntu and Debian; to fix this, we install vim-nox:

aptitude install vim-nox

(You don't have to do this if you use a different text editor such as joe or nano.)

 

 3. Configure The Network

Because the Ubuntu installer has configured our system to get its network settings via DHCP, we have to change that now because a server should have a static IP address. Edit /etc/network/interfaces and adjust it to your needs (in this example setup I will use the IP address 192.168.0.100):

vi /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.0.100
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1

Then restart your network:

/etc/init.d/networking restart

Then edit /etc/hosts. Make it look like this:

127.0.0.1       localhost.localdomain   localhost
192.168.0.100   server1.example.com     server1

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Now run:

echo server1.example.com > /etc/hostname
/etc/init.d/hostname.sh start

Afterwards, run:

hostname
hostname -f

Both should show server1.example.com now.

 

4. Update your Linux Installation

Edit /etc/apt/sources.list. Comment out or remove the installation CD from the file and make sure that the universe and multiverse repositories are enabled. It should look like this:

#
# deb cdrom:[Ubuntu-Server 9.04 _Jaunty Jackalope_ - Release amd64 (20090421.1)]/ jaunty main restricted

#deb cdrom:[Ubuntu-Server 9.04 _Jaunty Jackalope_ - Release amd64 (20090421.1)]/ jaunty main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://de.archive.ubuntu.com/ubuntu/ jaunty main restricted
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://de.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://de.archive.ubuntu.com/ubuntu/ jaunty universe
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty universe
deb http://de.archive.ubuntu.com/ubuntu/ jaunty-updates universe
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://de.archive.ubuntu.com/ubuntu/ jaunty multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty multiverse
deb http://de.archive.ubuntu.com/ubuntu/ jaunty-updates multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-updates multiverse

## Uncomment the following two lines to add software from the 'backports'
## repository.
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://de.archive.ubuntu.com/ubuntu/ jaunty-backports main restricted universe multiverse
# deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu jaunty partner
# deb-src http://archive.canonical.com/ubuntu jaunty partner

deb http://security.ubuntu.com/ubuntu jaunty-security main restricted
deb-src http://security.ubuntu.com/ubuntu jaunty-security main restricted
deb http://security.ubuntu.com/ubuntu jaunty-security universe
deb-src http://security.ubuntu.com/ubuntu jaunty-security universe
deb http://security.ubuntu.com/ubuntu jaunty-security multiverse
deb-src http://security.ubuntu.com/ubuntu jaunty-security multiverse

Then run the following to update the apt package database:

aptitude update

Run the following to install the latest updates:

aptitude safe-upgrade

If you see that a new kernel gets installed as part of the updates, you should reboot the system afterwards:

 

5. Change The Default Shell

/bin/sh is a symlink to /bin/dash, however we need /bin/bash, not /bin/dash. Therefore we do this:

dpkg-reconfigure dash

Install dash as /bin/sh? <-- No

 

6. Disable AppArmor

AppArmor is a security extension (similar to SELinux) that should provide extended security. In my opinion you don't need it to configure a secure system, and it usually causes more problems than advantages (think of it after you have done a week of trouble-shooting because some service wasn't working as expected, and then you find out that everything was ok, only AppArmor was causing the problem). Therefore I disable it (this is a must if you want to install ISPConfig later on).

We can disable it like this:

/etc/init.d/apparmor stop
update-rc.d -f apparmor remove
aptitude remove apparmor apparmor-utils

 

7. Install Some Software

Now we install a few packages that are needed later on:

aptitude install binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.6-dev libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ build-essential unrar

 

8. Synchronize the System Clock

It is a good idea to synchronize the system clock with an NTP (network time protocol) server over the internet. Simply run :

apt-get install ntp ntpdate

 

9. Caching Dnsmasq

apt-get install dnsmasq

Edit /etc/dnsmasq.conf and make Dnsmasq listen on localhost:

listen-address=127.0.0.1 

Edit /etc/resolv.conf and add the following to the top of the list:

nameserver 127.0.0.1 

Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Jamie Strandboge (not registered) on Mon, 2009-12-28 16:49.
I noticed that this tutorial recommends to disable all of AppArmor. Unless you have a very specific need to do so, this is not recommended. The apparmor profiles shipped in Ubuntu are designed to work with the default installation. If a particular profile is causing you trouble, please disable the profile or put it in complain mode, and leave the other profiles that are not causing problems to do their jobs. Better yet, file a bug. :) See my blog entry athttp://penguindroppings.wordpress.com/2009/07/07/should-i-disable-apparmor/ for details.
Submitted by peterwbowey (registered user) on Fri, 2009-06-26 17:25.

I was one of the likely many that recently updated Ubuntu/Debian PERL to the latest release of 5.10:

I found this latest PERL killed MailScanner - with logged (mail.log) reports of: "Insecure dependency in chown while running with the -T switch in ..../MailScanner/message.pm on line xxxx". MailScanner would then 'abort' any exec calls or just hang - leaving no outward flow of emails! Yet, after hours of research - there is a solution (other than going back to the older PERL V5.9.x)!

This new 'problem' results from the new PERL Taint Mode enforcement policy! This new 'rule' enables (and enforces) a number of wise security checks with programs (exec) called with different user and / or group ids.

There is a solution, and I have tested it with the latest release .tar of MailScanner v4.78.1:

In the Ubuntu / Debain's /etc/init.d/mailscanner startup script (or the default MailWasher /etc/rc2.d/S20mailscanner) to include the required UID / GUID on the daemon start line: (see next line)

Find the line in the Ubuntu /etc/init.d/mailscanner distro (or the orig MailScanners own .tar distro = /etc/rc2.d/S20mailscanner) that has:

start-stop-daemon --start --quiet --nicelevel $run_nice --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

and change it to:

start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

The effect of this change allows mailscanner to run with the required UID to prevent the new PERL enforced 'taint mode'. Please note that this means that MailWasher must have the UID set to postfix in the MailScanner.conf file.

Another change I found that was essential was to add the clamav user to the postfix group. Without this change I got permission denied errors when the clamd (daemon) was called by MailWasher to check emails on the /var/spool/postfix/incoming folder. (with UID = postfix).

With the new changes I have no further faults or errors using MailScanner (with all the extra 'SpamSnake' additions). This new code runs live on my own dedicated business mail server - so it is a live and real test! Normally, I used to recieve about 350 spam emails per day, now it has dropped to 2-3 that 'leak' through the 'SpamSnake'.

If any one is interested, I have ported the latest MailScanner tar release v4.78.1 to a true debain / ubuntu style package - without the generic /OPT/.... directory.

Peter Bowey

Submitted by Nivethan (not registered) on Mon, 2010-02-15 11:00.
Thank you verymuch, your method worked! I had been wondering with the mail scanner looping problem for two days and just before was going to go change back to a older perl version fortunately I found your comment! wow! brilliant....
Submitted by Frank Holler (not registered) on Mon, 2009-10-12 11:33.

Hi.

I am running Debian and mta exim4 and my error was different: "Insecure dependency in exec while running with -T switch at /usr/share/MailScanner//MailScanner/SweepOther.pm line 374"

So i changed  /etc/init.d/mailscanner and added "--chuid=Debian-exim" and the error disappeared. This helped.

Thanks alot.

Submitted by bdamon (registered user) on Thu, 2009-06-25 14:19.

After moving the newly installed MailScanner to /opt and renaming the repository installed version to /etc/MailScanner.dist  this leaves the MailScanner init script still configured to look for Mailscanner.conf in /etc/MailScanner. Either changing the init script or creating a link fixes this.

Submitted by peterwbowey (registered user) on Tue, 2009-06-23 05:02.

You see an error like this: (mail.log)

"ClamAV-autoupdate: ClamAV updater /usr/local/bin/freshclam cannot be run"

Solution: create a symbolic link from:   /usr/bin/freshclam -> /usr/local/bin/freshclam

Now the path reference works:

The [mail.log] will now show something like this:

"update.virus.scanners: Running autoupdate for clamav"
"ClamAV-autoupdate[31509]: ClamAV did not need updating"

Peter Bowey

Submitted by peterwbowey (registered user) on Mon, 2009-06-22 16:13.

These are live notes that I collected as I got this great Spam Snake working: (and do note that it works very well - when you overcome a 'few' problems)

The line with: [apt-get install mailscanner razor pyzor clamav-module] produces a problem finding the clamav-module: (missing resolution steps outlined)

Steps to get and install the [Spam Snake] clamav-module:

Be sure you have perl installed... (if not sure use: aptitude install perl) then enter:

perl -MCPAN -e shell

If you are prompted if you want to configure perl automatically choose: yes

When you are at [cpan] prompt enter (steps 1-7):

1) install CPAN (gets any upgrades for perl / cpan)
2) reload cpan (reload any new version)
3) test File::Scan::ClamAV (it will download the ClamAV module)
4) look File::Scan::ClamAV (shell to the ClamAV area)
5) make install (will perform the ClamAV-module install)
6) exit
7) quit

All done, now we have the ClamAV-Module!

UPDATE: You will likely find that continuing with the [SpamSnake] setup that there will be other missing PERL modules:

I outline the steps to resolve this (it was pure research) - and it now works:

Type: (at Linux command line)

cpan -i ExtUtils::Command::MM
aptitude install libconvert-binhex-perl
cpan -i Checker::ISA
cpan -i Archive::Zip
aptitude install libyaml-perl
cpan -i OLE::Storage_Lite


Now we are almost there: - see if it now works OK

Test the final perl setup with this:

/opt/MailScanner/bin/check_mailscanner

The given steps (in this Spam Snake guide) shown as:

Create /etc/postfix/relay_recipients and add the following:

@example.com OK
@example2.com OK

Create /etc/postfix/transport and add the following:

example.com smtp:[192.168.0.x]
example2.com smtp:[192.168.0.x]

Create /etc/postfix/relay_domains and add the following:

example.com OK
example2.com OK

Must NOT include any reference to any declared virtual email domains, or you will see this error:

"postfix/trivial-rewrite: warning: do not list domain mydomain.com in BOTH virtual_mailbox_domains and relay_domains"

The line that has: example.com smtp:[192.168.0.x] must be a full and valid IP address!


I have performed several email tests and I am very happy with the final process, Additionally, all outgoing emails show this text report:

This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Peter Bowey

Submitted by Miguel (registered user) on Fri, 2009-12-11 06:36.

In the Ubuntu / Debain's /etc/init.d/mailscanner startup script (or the default MailWasher /etc/rc2.d/S20mailscanner) to include the required UID / GUID on the daemon start line: (see next line)

Find the line in the Ubuntu /etc/init.d/mailscanner distro (or the orig MailScanners own .tar distro = /etc/rc2.d/S20mailscanner) that has:

start-stop-daemon --start --quiet --nicelevel $run_nice --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

and change it to:

start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

 This doesn't work for me since I run Karmic. I get an error that Mailscanner can't set the gid in ... line ...

Solution:

start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix:www-data --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

Runs like a charm now with no errors.