The Perfect SpamSnake - Ubuntu Jaunty Jackalope - Page 2

10. Apache2

apt-get install apache2 apache2-suexec apache2-doc apache2-mpm-prefork apache2-utils libexpat1

Next we install PHP5 as an Apache module:

aptitude install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick php5-imap php5-mcrypt php5-memcache php5-mhash php5-ming php5-mysql php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl

Next we edit /etc/apache2/mods-available/dir.conf and change the DirectoryIndex line:

<IfModule mod_dir.c>

          #DirectoryIndex index.html index.cgi index.pl index.php index.xhtml index.htm
          DirectoryIndex index.html index.htm index.shtml index.cgi index.php index.php3 index.pl index.xhtml

</IfModule>

Now we have to enable some Apache modules (rewrite, suexec, include)

a2enmod rewrite
a2enmod suexec
a2enmod include

Restart Apache:

/etc/init.d/apache2 restart

11. Postfix and MySQL

Install the packages:

apt-get install postfix postfix-mysql postfix-doc mysql-server mysql-client procmail

MySQL:

You will be asked to provide a password for the MySQL root user - this password is valid for the user root@localhost as well as root@server1.example.com, so we don't have to specify a MySQL root password manually later on:

New password for the MySQL "root" user: <-- yourrootsqlpassword
Repeat password for the MySQL "root" user: <-- yourrootsqlpassword

Postfix:

You will be asked two questions. Answer as follows:

General type of mail configuration: <-- Internet Site
System mail name: <-- server1.example.com

We want MySQL to listen on all interfaces, not just localhost, therefore we edit /etc/mysql/my.cnf and comment out the line bind-address = 127.0.0.1:

[...]
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
[...]

Then we restart MySQL:

/etc/init.d/mysql restart

Now check that networking is enabled. Run

netstat -tap | grep mysql

The output should look like this:

tcp        0      0 *:mysql                 *:*                     LISTEN      4318/mysqld

Stop Postfix:

postfix stop

We'll want to edit Postfix with the below:

Edit master.cf:

We need to add two items below the pickup service type. The pickup service "picks up" local mail (local meaning "on this machine") and delivers it. This is a way to bypass content filtering for mail generated by this machine.

It should look like this when you are done:

pickup    fifo  n       -       -       60      1       pickup
         -o content_filter=
         -o receive_override_options=no_header_body_checks

Edit main.cf:

postconf -e "alias_maps = hash:/etc/aliases"
newaliases
postconf -e "myorigin = example.com"
postconf -e "myhostname = server1.example.com"
postconf -e "mynetworks = 127.0.0.0/8, 192.168.0.0/24"
postconf -e "message_size_limit = 10485760"
postconf -e "local_transport = error:No local mail delivery"
postconf -e "mydestination = "
postconf -e "local_recipient_maps = "
postconf -e "virtual_alias_maps = hash:/etc/postfix/virtual"

Create /etc/postfix/virtual and add the following:

postmaster postmaster@example.com
abuse abuse@example.com
root root@example.com

Continue:

postmap /etc/postfix/virtual
postconf -e "relay_recipient_maps = hash:/etc/postfix/relay_recipients"

Create /etc/postfix/relay_recipients and add the following:

@example.com OK
@example2.com OK

Continue:

postmap /etc/postfix/relay_recipients
postconf -e "transport_maps = hash:/etc/postfix/transport"

Create /etc/postfix/transport and add the following:

example.com smtp:[192.168.0.x]
example2.com smtp:[192.168.0.x]

Continue:

postmap /etc/postfix/transport
postconf -e "relay_domains = hash:/etc/postfix/relay_domains"

Create /etc/postfix/relay_domains and add the following:

example.com OK
example2.com OK

Continue:

postmap /etc/postfix/relay_domains
postconf -e "smtpd_helo_required = yes"
postconf -e "smtpd_sender_restrictions = reject_non_fqdn_sender, reject_unknown_sender_domain, permit"
postconf -e "smtpd_recipient_restrictions = reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_recipient_domain, permit_mynetworks, reject_unauth_destination, reject_unauth_pipelining, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, permit"
postconf -e "smtpd_data_restrictions = reject_unauth_pipelining"

Let's take a final look at the Postfix configuration:

less /etc/postfix/main.cf

Check the contents of the file for errors and repair if needed. Fire up Postfix:

postfix start

Check that Postfix responds:

telnet 127.0.0.1 25

You should see:

220 [yourFQDNhere] ESMTP Postfix (Ubuntu)

12. MailScanner Razor Pyzor DCC Clamav Installation

apt-get install mailscanner razor pyzor clamav-daemon

Let's start with MailScanner. The MailScanner that was just installed from the repositories is a very old version so we will now remove it and install the MailScanner package from source. The above is done to install the dependencies only.

apt-get remove mailscanner

Download http://www.mailscanner.info/files/4/tar/MailScanner-install-4.75.11-1.tar.gz into /usr/src/mailscanner/ and run:

tar xvfz MailScanner-install-4.75.11-1.tar.gz
cd MailScanner-install-4.75.11
./install.sh

Disable the default MailScanner:

mv /etc/MailScanner /etc/MailScanner.dist

Share this page:

32 Comment(s)

Add comment

Comments

From: at: 2009-06-22 15:13:39

These are live notes that I collected as I got this great Spam Snake working: (and do note that it works very well - when you overcome a 'few' problems)

The line with: [apt-get install mailscanner razor pyzor clamav-module] produces a problem finding the clamav-module: (missing resolution steps outlined)

Steps to get and install the [Spam Snake] clamav-module:

Be sure you have perl installed... (if not sure use: aptitude install perl) then enter:

perl -MCPAN -e shell

If you are prompted if you want to configure perl automatically choose: yes

When you are at [cpan] prompt enter (steps 1-7):

1) install CPAN (gets any upgrades for perl / cpan)
2) reload cpan (reload any new version)
3) test File::Scan::ClamAV (it will download the ClamAV module)
4) look File::Scan::ClamAV (shell to the ClamAV area)
5) make install (will perform the ClamAV-module install)
6) exit
7) quit

All done, now we have the ClamAV-Module!

UPDATE: You will likely find that continuing with the [SpamSnake] setup that there will be other missing PERL modules:

I outline the steps to resolve this (it was pure research) - and it now works:

Type: (at Linux command line)

cpan -i ExtUtils::Command::MM
aptitude install libconvert-binhex-perl
cpan -i Checker::ISA
cpan -i Archive::Zip
aptitude install libyaml-perl
cpan -i OLE::Storage_Lite


Now we are almost there: - see if it now works OK

Test the final perl setup with this:

/opt/MailScanner/bin/check_mailscanner


The given steps (in this Spam Snake guide) shown as:

Create /etc/postfix/relay_recipients and add the following:

@example.com OK
@example2.com OK

Create /etc/postfix/transport and add the following:

example.com smtp:[192.168.0.x]
example2.com smtp:[192.168.0.x]

Create /etc/postfix/relay_domains and add the following:

example.com OK
example2.com OK

Must NOT include any reference to any declared virtual email domains, or you will see this error:

"postfix/trivial-rewrite: warning: do not list domain mydomain.com in BOTH virtual_mailbox_domains and relay_domains"

The line that has: example.com smtp:[192.168.0.x] must be a full and valid IP address!


I have performed several email tests and I am very happy with the final process, Additionally, all outgoing emails show this text report:

This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Peter Bowey

From: at: 2009-12-11 05:36:56

In the Ubuntu / Debain's /etc/init.d/mailscanner startup script (or the default MailWasher /etc/rc2.d/S20mailscanner) to include the required UID / GUID on the daemon start line: (see next line)

Find the line in the Ubuntu /etc/init.d/mailscanner distro (or the orig MailScanners own .tar distro = /etc/rc2.d/S20mailscanner) that has:

start-stop-daemon --start --quiet --nicelevel $run_nice --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

and change it to:

start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

 This doesn't work for me since I run Karmic. I get an error that Mailscanner can't set the gid in ... line ...

Solution:

start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix:www-data --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

Runs like a charm now with no errors.

 

From: at: 2009-06-23 04:02:04

You see an error like this: (mail.log)

"ClamAV-autoupdate: ClamAV updater /usr/local/bin/freshclam cannot be run"

Solution: create a symbolic link from:   /usr/bin/freshclam -> /usr/local/bin/freshclam

Now the path reference works:

The [mail.log] will now show something like this:

"update.virus.scanners: Running autoupdate for clamav"
"ClamAV-autoupdate[31509]: ClamAV did not need updating"

Peter Bowey

From: at: 2009-06-25 13:19:42

After moving the newly installed MailScanner to /opt and renaming the repository installed version to /etc/MailScanner.dist  this leaves the MailScanner init script still configured to look for Mailscanner.conf in /etc/MailScanner. Either changing the init script or creating a link fixes this.

From: at: 2009-06-26 16:25:59

I was one of the likely many that recently updated Ubuntu/Debian PERL to the latest release of 5.10:

I found this latest PERL killed MailScanner - with logged (mail.log) reports of: "Insecure dependency in chown while running with the -T switch in ..../MailScanner/message.pm on line xxxx". MailScanner would then 'abort' any exec calls or just hang - leaving no outward flow of emails! Yet, after hours of research - there is a solution (other than going back to the older PERL V5.9.x)!

This new 'problem' results from the new PERL Taint Mode enforcement policy! This new 'rule' enables (and enforces) a number of wise security checks with programs (exec) called with different user and / or group ids.

There is a solution, and I have tested it with the latest release .tar of MailScanner v4.78.1:

In the Ubuntu / Debain's /etc/init.d/mailscanner startup script (or the default MailWasher /etc/rc2.d/S20mailscanner) to include the required UID / GUID on the daemon start line: (see next line)

Find the line in the Ubuntu /etc/init.d/mailscanner distro (or the orig MailScanners own .tar distro = /etc/rc2.d/S20mailscanner) that has:

start-stop-daemon --start --quiet --nicelevel $run_nice --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

and change it to:

start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

The effect of this change allows mailscanner to run with the required UID to prevent the new PERL enforced 'taint mode'. Please note that this means that MailWasher must have the UID set to postfix in the MailScanner.conf file.

Another change I found that was essential was to add the clamav user to the postfix group. Without this change I got permission denied errors when the clamd (daemon) was called by MailWasher to check emails on the /var/spool/postfix/incoming folder. (with UID = postfix).

With the new changes I have no further faults or errors using MailScanner (with all the extra 'SpamSnake' additions). This new code runs live on my own dedicated business mail server - so it is a live and real test! Normally, I used to recieve about 350 spam emails per day, now it has dropped to 2-3 that 'leak' through the 'SpamSnake'.

If any one is interested, I have ported the latest MailScanner tar release v4.78.1 to a true debain / ubuntu style package - without the generic /OPT/.... directory.

Peter Bowey

From: Frank Holler at: 2009-10-12 10:33:20

Hi.

I am running Debian and mta exim4 and my error was different: "Insecure dependency in exec while running with -T switch at /usr/share/MailScanner//MailScanner/SweepOther.pm line 374"

So i changed  /etc/init.d/mailscanner and added "--chuid=Debian-exim" and the error disappeared. This helped.

Thanks alot.

From: Nivethan at: 2010-02-15 10:00:48

Thank you verymuch, your method worked! I had been wondering with the mail scanner looping problem for two days and just before was going to go change back to a older perl version fortunately I found your comment! wow! brilliant....

From: Jamie Strandboge at: 2009-12-28 15:49:00

I noticed that this tutorial recommends to disable all of AppArmor. Unless you have a very specific need to do so, this is not recommended. The apparmor profiles shipped in Ubuntu are designed to work with the default installation. If a particular profile is causing you trouble, please disable the profile or put it in complain mode, and leave the other profiles that are not causing problems to do their jobs. Better yet, file a bug. :) See my blog entry athttp://penguindroppings.wordpress.com/2009/07/07/should-i-disable-apparmor/ for details.

From: at: 2009-06-28 03:30:11

Update for Ubuntu / Debian:

The default debian system does not have a /var/lock/susbsys, and in a normal chroot - it needs to be created before the MailWasher Daemon starts:

I have amended the required change => /etc/rc2.d/S20mailscanner

16.2 Fix to Disable Permission Checks on MailScanner Directories

Edit /etc/rc2.d/S20mailscanner to look like:

check_dir /var/spool/MailScanner       ${user:-postfix} ${group:-postfix}
check_dir /var/lib/MailScanner         ${user:-postfix} ${group:-postfix}
check_dir /var/run/MailScanner         ${user:-postfix} ${group:-postfix}
check_dir /var/lock/subsys             ${user:-root}    ${group:-root}
check_dir /var/lock/subsys/MailScanner ${user:-postfix} ${group:-postfix}
In the file /etc/default/mailscanner, make sure this parameter is at 1:
 
The logic of the above is made apparent when you see the rest of the /etc/rc2.d/S20mailscanner file contents:

check_dir()
{
    if [ ! -d $1 ]; then
        mkdir -p "$1" || \
            fail "directory $1: does not exist and cannot be created"
    fi
    actual="$(stat -c %U $1)"
    if [ "$actual" != "$2" ]; then
        chown -R "$2" "$1" || \
            fail "directory $1: wrong owner (expected $2 but is $actual)"
    fi
    actual="$(stat -c %G $1)"
    if [ "$actual" != "$3" ]; then
        chgrp -R "$3" "$1" || \
            fail "directory $1: wrong group (expected $3 but is $actual)"
    fi
}
user=$(echo $(awk -F= '/^Run As User/ {print $2; exit}' $CONFFILE))
group=$(echo $(awk -F= '/^Run As Group/ {print $2; exit}' $CONFFILE))
check_dir /var/spool/MailScanner       ${user:-postfix} ${group:-postfix}
check_dir /var/lib/MailScanner         ${user:-postfix} ${group:-postfix}
check_dir /var/run/MailScanner         ${user:-postfix} ${group:-postfix}
check_dir /var/lock/subsys             ${user:-root}    ${group:-root}  ## required to CREATE folder!
check_dir /var/lock/subsys/MailScanner ${user:-postfix} ${group:-postfix}


The above change avoids the need to dynamically add these directories through the slower use of /etc/rc.local (as I have seen Ubuntu / Debian MailScanner use as a 'solution' when they discovered that the MailScanner ( /var/xxx folders ) were lost after a reboot.

A lot of debian / ubuntu users use enhanced protection to the /var/xxx folder areas, and then it is normal to have to create 'real' chroot access across reboots.”

Spamsnake's author mentions the need to install clamav-module; this is an older perl module that is superseded by Ubuntu's clamav-daemon. The author is actually using this new daemon for mailwasher - evident by the
MailWasher.conf file with:

Virus Scanners = clamd                     ## clamav daemon use


So it not required to chase the older perl clamav-module; just get the latest Ubuntu clamav-daemon

aptitude install clamav-daemon
 

Notes: The clamav-daemon is faster than the previous perl
clamav-module! MailSanners own wiki states this change and recommended use.

Peter Bowey

From: Anonymous at: 2009-09-02 22:32:43

I got this error after I do postmap /etc/postfix virtual:

postmap: fatal: /etc/postfix/main.cf, line 43: missing '=' after attribute name:  "postconf -e alias_maps = hash:etc/aliases"

 

From: Anonymous at: 2009-07-31 03:50:25

Hello

Just letting you know that DCC has been removed from all the debian apt sources.

There was a security bug which was causing issues to the dcc wich are unfixable

 see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464161

 

From: Anonymous at: 2010-08-05 18:47:55

This bug was fixed in dcc 1.3.51 - whereas this article describes installing 1.3.103.

 So... not an issue.

From: Anonymous at: 2009-08-03 02:04:36

Hello

The latest version of mailscanner is only compatable with mailtools 1.77

Which I couldnt find in apt, I had to install it manually following this

http://cpansearch.perl.org/src/MARKOV/MailTools-1.77/README

From: at: 2009-08-10 16:29:57

Before step 16.4 where you start MailScanner, you will need to set the permissions on the spool folders like this:

chown -R postfix:www-data MailScanner/

From: at: 2009-08-10 16:22:44

At step 13, it is mentioned that we need to install Python 2.5, but never specifically in the code blocks. In case anyone gets stuck, don't forget to run this before you do step 13:

apt-get install python2.5

From: citybird at: 2009-10-23 13:11:37

 the following setting should also be changed in conf.php in step 17.5

define(MS_CONFIG_DIR, '/opt/MailScanner/etc/');

From: bearman at: 2009-09-25 06:27:32

We got it!

It works if Y substitute:

---------------------------------------------------------

with:

--

:-)

 

From: nibb13 at: 2009-09-11 18:28:18

Try changing line 4 of create.sql to read:

-- -------------------------------------

That seems to have done the trick for me.

Cheers, nibb13

From: at: 2010-02-18 02:28:24

This is one of the most interesting and useful "how tos", but extremely unprofessionally done and has lots of bugs. I say thank you to the author, but please read the comments and edit the steps and missteps. 

 Also, some steps are extremely for kids who don't know computers, but some steps you say that require extra research that expert users in computers, but not in linux won't be able to follow.

 Also to make it easier, when you say "edit this file" you can simpler provide a command "vi/nano /etc/blahblahblah"

 Now, step 17.9, a little lower you say "Edit the SpamAssassin v310.pre to enable Razor and DCC"
where is it? where do I find it to edit?

From: Jim Morbid at: 2010-03-31 13:17:23

joe /etc/mail/spamassassin/v310.pre

 

:-)

From: ScarEye at: 2009-09-06 02:49:04

17.2 is kicking my ass this what I get when I run

mysql -p < create.sql

root@smtpgw:/usr/src/mailwatch-1.0.4# mysql -p < create.sql
Enter password:
ERROR 1064 (42000) at line 4: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-------------------------------------------------------

 

 


CREATE DATABASE /*' at line 1
root@smtpgw:/usr/src/mailwatch-1.0.4#

Here is a few lines of create.sql

-- MySQL dump 8.23
--
-- Host: localhost    Database: mailscanner
---------------------------------------------------------
-- Server version       3.23.58

--
-- Current Database: mailscanner
--

CREATE DATABASE /*!32312 IF NOT EXISTS*/ mailscanner;

USE mailscanner;

--
-- Table structure for table `audit_log`
--

CREATE TABLE audit_log (
  timestamp timestamp(14) NOT NULL,
  user varchar(20) NOT NULL default '',
  ip_address varchar(15) NOT NULL default '',
  action text NOT NULL
) TYPE=MyISAM;

--
-- Table structure for table `blacklist`
--

CREATE TABLE blacklist (
  id int(11) NOT NULL auto_increment,
  to_address text,
  to_domain text,
  from_address text,
  PRIMARY KEY  (id),
  UNIQUE KEY blacklist_uniq (to_address(100),from_address(100))
) TYPE=MyISAM;

--
-- Table structure for table `geoip_country`
--

CREATE TABLE geoip_country (
  begin_ip varchar(15) default NULL,
  end_ip varchar(15) default NULL,
  begin_num bigint(20) default NULL,
  end_num bigint(20) default NULL,
  iso_country_code char(2) default NULL,
  country text,
  KEY geoip_country_begin (begin_num),
  KEY geoip_country_end (end_num)
) TYPE=MyISAM;

--

 

I ran a few lines manually and they seem to work.  Then I tried to re-run mysql -p < create.sql

but no luck.

Please help, so close to being done.

 

Thanks

ScarEye

From: at: 2014-04-01 11:42:49

Hi, 

 To fix the error 1064 with this setup on a new system, just remove the (14) on the timestamp timestamp entries in create.sql

 There are 2 one at the top and one on line 78 (to get there in nano press CTRL _)

From: ScarEye at: 2010-01-12 15:02:55

Anyone here know how to fix the 2010 bug with spamassassin?  I see this here.  Check this out.

 http://wiki.apache.org/spamassassin/Rules/FH_DATE_PAST_20XX

 But in spamsnake we disbale local.cf, so what's the work around?

 

Thanks,

ScarEye

From: Anonymous at: 2010-08-05 18:55:41

In step 17.10, I ran into an error when trying to run:

 mysql -u sa_user -p sa_bayes < /path/to/bayes_mysql.sql

A simple fix is to add the following to the top of the .sql file:

 CONNECT sa_bayes;

 

From: shawn at: 2009-06-11 18:03:41

can this spam snake be configured to scan and relay to multiple mail hosts ?

From: randomxs at: 2009-06-14 04:45:29

There's no reason why it can't. I do it at work for multiple domains and mail servers.

From: Martin H at: 2009-07-15 20:06:12

And which changes you did to postfix configuration in order to make it work with multiple domains / mail servers??

Thanks

 Martin

From: Matt at: 2009-07-20 21:25:05

Add the domains to relay_domains, relay_recipients, and transport.

From: Anonymous at: 2009-06-13 07:33:08

I stopped reading at 8.:

 1. Statistically, brute force attack now has 100% bigger chances on guessing 'administrator' password. It's easier to guess one of two, than only one.

2. ubuntu server comes with vim installed by default. vim-nox is added support for perl, python and ruby.

5. why would you change default shell? If the scripts are broken, fix them; don't avoid the problem.

6. Now, that's a stupid thing to do. Again, as with 5, why not rather fix the problem, instead of avoiding it?

8. ntpdate is installed by default, and you don't need it if you have ntp service running. Hell, you can't even use ntpdate while ntpd is running.

From: Anonymous at: 2010-01-26 04:01:00

To run ntpdate while ntp is running simply use the -u switch.

From: maxsec at: 2009-06-14 19:29:12

you might want to to put in extra SA rules and also turn off many of the RBL's in SA.

 Also watch out for the spamlist settings in MailScanner.conf - usually better to do  this in SA rather than MS. The Spamhaus lists (Zen etc) can also block you if you are querying then alot - see their TOS on this.

 

Might be worth pointing at the performance and "Getting the most out of spamassassin" sections of the MailScanner Wiki

 

 

From: Sebastian at: 2009-12-20 15:10:54

Hi guys,

 you'd better issue

sa-learn --sync -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf

 before doing

spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf --lint

This way you are initializing the database and after that the testout will succeed.

 

Best regards,

Sebastian