The Perfect SpamSnake - Ubuntu Jeos 10.10 Maverick Meerkat - Page 3

8. Baruwa

Download custom baruwa deb package:

cd /usr/src
wget baruwa_1.0.1-2sn_all.deb Updated 021811
wget baruwa-doc_1.0.1-2_all.deb
gdebi baruwa*.deb

You'll be prompted to setup the baruwa db for logging, including an access username and password, which will be used by the Baruwa scripts. You'll also be prompted to setup the admin user information for the Baruwa frontend.

vi /usr/share/pyshared/baruwa/settings.py

Change the Quarantine hosturl to your liking:

QUARANTINE_REPORT_HOSTURL = 'http://baruwa-alpha.local'

 

vi /opt/MailScanner/etc/MailScanner.conf

You need to make sure that the following options are set:

Always Looked Up Last = &BaruwaSQL
Is Definitely Not Spam = &BaruwaWhitelist
Is Definitely Spam = &BaruwaBlacklist
Required SpamAssassin Score = &BaruwaLowScore
High SpamAssassin Score = &BaruwaHighScore 

 

Upgrading Baruwa

*Note: This section is only for those running an older version of Baruwa. Skip this if you're doing a fresh install.

Download the latest baruwa deb package:

cd /usr/src
wget baruwa_1.0.2-4sn_all.deb Updated 052011
wget baruwa-doc_1.0.2-4_all.deb
gdebi baruwa*.deb

When prompted to setup the mysql db, select no. We'll reuse the current db and settings.

Make sure /usr/share/pyshared/baruwa/settings.py is setup with the correct database information before you connect, otherwise your connection will fail.

vi /usr/share/pyshared/baruwa/settings.py

DATABASE_ENGINE = 'mysql'           # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
DATABASE_NAME = 'baruwa'             # Or path to database file if using sqlite3.
DATABASE_USER = 'baruwa'             # Not used with sqlite3.
DATABASE_PASSWORD = 'password'         # Not used with sqlite3.
DATABASE_HOST = ''             # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = ''             # Set to empty string for default. Not used with sqlite3.

 

*Note: Due to a few minor upgrade conflicts, these steps might have to be taken to correctly upgrade baruwa.

If /opt/MailScanner was removed:

ln -s /opt/MailScanner-version /opt/MailScanner
rm -r /etc/MailScanner
ln -s /opt/MailScanner/etc /etc/MailScanner
ln -s /opt/MailScanner/lib/MailScanner/CustomFunctions /etc/MailScanner

Make sure to update your database connection string in /etc/MailScanner/CustomFunctions/*.pm files.

Finally, run

manage.py syncdb

Restart Uwsgi and Nginx

/etc/init.d/uwsgi-python2.6 restart && /etc/init.d/nginx restart

Baruwa should be upgraded to 102-4sn.

 

Upgrading Baruwa 102-4 to 111-3sn

Install Rabbitmq-Server 2.2.0 and setup db/user/password:

cd /usr/src
wget http://ppa.launchpad.net/drizzle-developers/ppa/ubuntu/pool/main/r/rabbitmq-server/rabbitmq-server_2.2.0-1~maverick0_all.deb
gdebi rabbit*

vi /etc/rabbitmq/rabbitmq.conf to bind to localhost:

export RABBITMQ_NODENAME=rabbit@localhost
export RABBITMQ_NODE_IP_ADDRESS=127.0.0.1
export ERL_EPMD_ADDRESS=127.0.0.1

Add the database credentials:

rabbitmqctl add_user baruwa password
rabbitmqctl add_vhost baruwa
rabbitmqctl set_permissions -p baruwa baruwa ".*" ".*" ".*"

Restart rabbitmq-server:

/etc/init.d/rabbitmq-server restart

Add sources for Baruwa 1.1.1-3 and install dependencies:

wget -O - http://apt.baruwa.org/baruwa-apt-keys.gpg | apt-key add -

vi /etc/apt/sources.list

#baruwa
deb http://apt.baruwa.org/ubuntu maverick main

Install Dependencies:

apt-get update
apt-get install python-django-celery python-importlib

*Note: The above command should install all dependencies for Baruwa.

Download and install Baruwa1.1.1-3sn:

mkdir /usr/src/baruwa1113 && cd /usr/src/baruwa1113
wget https://docs.google.com/open?id=0B9cN15Q3pKnwY2YyMmMwOTQtNGJkMi00ZDc2LWFjMjMtY2UxMzFlYWY5Mzk2
mv open* baruwa_1.1.1-3sn_all.deb
wget https://docs.google.com/open?id=0B9cN15Q3pKnwODk2OGViYmMtMGZmYS00NmJjLTkwZWUtNTJjYTQ1YzAzOTg1
mv open* baruwa-doc_1.1.1-3sn_all.deb
gdebi baruwa_1.1.1-3sn_all.deb
gdebi baruwa-doc_1.1.1-3sn_all.deb

*Note: Install package maintainer’s version of files when asked. Select no to setup mysql as we’re upgrading from a previous version.

Fix the symlink for settings.py:

rm –r /usr/share/pyshared/baruwa/settings.py && ln –s /etc/baruwa/settings.py /usr/share/pyshared/baruwa/

vi /etc/baruwa/settings.py and fix the baruwa database configuration:
DATABASES = {
    'default': {
        # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3'
        # or 'oracle'.
        'ENGINE': 'django.db.backends.mysql',
        # Or path to database file if using sqlite3.
        'NAME': 'baruwa',
        # Not used with sqlite3.
        'USER': 'baruwa',
        # Not used with sqlite3.
        'PASSWORD': 'password',
        # Set to empty string for localhost. Not used with sqlite3.
        'HOST': '',
        # Set to empty string for default. Not used with sqlite3.
        'PORT': '',
    }
}

 

Update Baruwa database structure:

baruwa-admin syncdb --noinput

Run fake migration to initiate an upgrade:

for name in $(echo "accounts messages lists reports status config"); do
baruwa-admin migrate $name 0001 --fake;
done

Run the normal migration:

for name in $(echo "accounts messages lists reports status fixups config"); do
baruwa-admin migrate $name;
done

vi /etc/MailScanner/MailScanner.conf change the following:

Run As Group = celeryd
Quarantine User = celeryd
Quarantine Group = celeryd

vi /etc/MailScanner/conf.d/baruwa.conf:

Quarantine User = postfix #(Or what ever your `Run As User` is set to)
DB DSN = DBI:mysql:database=baruwa;host=localhost;port=3306 #set to valid DSN
DB Username = baruwa # your DB username
DB Password = password # your DB password

vi /etc/init.d/mailscanner and change group entries from www-data to celeryd:

check_dir /var/spool/MailScanner ${user:-postfix} ${group:-celeryd}
check_dir /var/lib/MailScanner ${user:-postfix} ${group:-celeryd}
check_dir /var/run/MailScanner ${user:-postfix} ${group:-celeryd}
check_dir /var/lock/subsys ${user:-root} ${group:-root} #Required to Create Folder
check_dir /var/lock/subsys/MailScanner ${user:-postfix} ${group:-celeryd}
start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix:celeryd --exec $DAEMON --name $NAME -- $DAEMON_ARGS \

Add celeryd user to clamav group:

usermod -a -G celeryd clamav

Change the group ownership of the quarantine folder and content:

chgrp -R celeryd /var/spool/MailScanner/quarantine

If you’re going to use signatures, initialize it by running:

baruwa-admin initconfig

Otherwise, disable the options is /etc/MailScanner/conf.d/baruwa.conf:

#Inline HTML Signature = htmlsigs.customize
#Inline Text Signature = textsigs.customize
#Signature Image Filename = sigimgfiles.customize
#Signature Image Filename = sigimgs.customize

Reboot your system and enjoy Baruwa 111-3sn.

 

9. Nginx with Uwsgi

Download and install both Nginx and Uwsgi from Chris Lea’s Launchpad:

wget https://launchpad.net/~chris-lea/+archive/nginx-devel/+files/nginx_1.0.0-1chl1%7Emaverick1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb
wget https://launchpad.net/~chris-lea/+archive/uwsgi/+files/uwsgi-common_0.9.6.6-1chl1%7Emaverick1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb
wget https://launchpad.net/~chris-lea/+archive/uwsgi/+files/uwsgi-extra_0.9.6.6-1chl1%7Emaverick1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb
wget https://launchpad.net/~chris-lea/+archive/uwsgi/+files/uwsgi-python2.6_0.9.6.6-1chl1%7Emaverick1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb
wget https://launchpad.net/~chris-lea/+archive/uwsgi/+files/uwsgi_0.9.6.6-1chl1%7Emaverick1_all.deb
apt-get install libsctp1
dpkg -i nginx*
dpkg -i uwsgi*

Use the following config files for uwsgi and nginx:

vi /etc/uwsgi/uwsgi-python2.6/baruwa.ini

[uwsgi]
   socket = /var/run/uwsgi/uwsgi-python2.6/baruwa/baruwa.sock
   pythonpath = /usr/share/pyshared/baruwa/
   master = true
   processes = 2
   env = DJANGO_SETTINGS_MODULE=baruwa.settings
   module = django.core.handlers.wsgi:WSGIHandler()

vi /etc/nginx/sites-available/baruwa.conf

   server {
      listen 80;
      server_name example.com;
      root /usr/share/pyshared/baruwa;
      #main access log
      access_log  /var/log/nginx/access.log;
      #main error log
      error_log /var/log/nginx/error.log;
   location /static {
      root  /usr/share/pyshared/baruwa/static/;
      }
   # static resources
   location ~* ^.+\.(html|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$
      {
      expires 30d;
      break;
      }
   location / {
      uwsgi_pass unix:///var/run/uwsgi/uwsgi-python2.6/baruwa/baruwa.sock;
      include uwsgi_params;
      }
   }

*Note: Make sure to change example.com to a FQDN that matches your setup.

Remove the default virtual host and copy over uwsgi_params:

rm -r /etc/nginx/sites-enabled/default
cp /usr/share/doc/uwsgi-extra/nginx/uwsgi_params /etc/nginx/uwsgi_params
ln -s /etc/nginx/sites-available/baruwa.conf /etc/nginx/sites-enabled/baruwa.conf

Restart services:

/etc/init.d/uwsgi-python2.6 restart && /etc/init.d/nginx restart

You should be all set.

Set up Cron jobs

Create a symlink to manage.py:

ln -s /usr/share/pyshared/baruwa/manage.py /usr/bin/manage.py
chmod +x /usr/bin/manage.py

Add cron jobs:

@daily  manage.py cleanquarantine &> /dev/null  #Clean quarantine
@daily  manage.py sendquarantinereports &> /dev/null  #Send quarantine reports
@monthly manage.py  dbclean &> /dev/null  #Clean maillog
@weekly  manage.py updatesarules &> /dev/null  #Update spamassassin rules
@daily  manage.py sendpdfreports &> /dev/null #Send PDF Reports

Start up MailScanner:

/etc/init.d/mailscanner start

Point your browser to http://hostname_used login with admin user and password and start working. You can now use the interface to add users and process messages, etc.

 

Setup Instructions for Baruwa

Log into Baruwa as admin --> Settings --> Accounts --> Create Account

Once you've created the user account, you'll get two new tabs on that page, Profile Settings and Associated Addresses.

Fill out Profile Settings choosing Domain Admin and set a low score of 6 and a high score of 9 and check scan email.

Click the + sign under Associated Addresses and enter a domains for which the user is the admin of eg. domain.com. domain.com will show up under Associated Addresses.

Click on domain.com and you'll be taken into Domain Information, where you'll be able to setup SMTP delivery information. Go ahead and add the receiving smtp server, or the ip of the receiving smtp server. Select enable and if you use a non-standard port, set it, otherwise use 25. Once you've done that, you can click on the test button next to the pencil, to see if your receiving server will accept the connection.

Now, you can log out as admin, and log in as the user you just setup and mails should start flowing.

The relay_recipients, relay_domains and transports settings in /etc/postfix/main.cf will use the entries you've provided in Baruwa. Therefore, no hash file is required.
The associated queries in the mysql cf files will pull the result in the proper format and feed it to postfix for use.

However, if you'd like to use a hash for any of your config files, use the following as an example of how to setup /etc/postfix/main.cf:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

Of course, you would have to create the hash file(s), populate it and postmap it for postfix to use.

*Note: If you do end up using hash for relay_recipients for specific domains, you'll have to remove that domain from /etc/postfix/access.  All other domain users can still be verified using look_ahead

Also, if you need to do mx lookups, you'll have to edit /etc/postfix/mysql-transports.cf query to look like:

concat('smtp:', mail_hosts.address, ':', port) 'transport'
The [ and ] were removed to allow MX lookups.

Enjoy Baruwa!

 

10. Install and Configure SPF

The postfix-policyd-spf-perl package depends on the Mail::SPF and the NetAddr::IP Perl modules.

We need to download postfix-policyd-spf-perl from http://www.openspf.org/Software to the /usr/src/ directory and install it to the /usr/lib/postfix/ directory like this:

cd /usr/src
wget http://www.openspf.org/blobs/postfix-policyd-spf-perl-2.007.tar.gz
tar xvfz postfix-policyd-spf-perl-2.007.tar.gz
cd postfix-policyd-spf-perl-2.007
cp postfix-policyd-spf-perl /usr/lib/postfix/policyd-spf-perl

Then we edit /etc/postfix/master.cf and add the following stanza at the end:

vi /etc/postfix/master.cf

policy unix - n n - - spawn
  user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl

(The leading spaces before user=nobody are important so that Postfix knows that this line belongs to the previous one!)

*Note: We already added the entry for main.cf using the postfix setup script.

Then restart Postfix:

/etc/init.d/postfix restart

That's it already.

 

11. Install and Configure FuzzyOcr

FuzzyOCR has some prerequisites like ocrad and gocr that we can install like this:

apt-get install fuzzyocr netpbm gifsicle libungif-bin gocr ocrad libstring-approx-perl libmldbm-sync-perl libdigest-md5-perl libdbd-mysql-perl imagemagick tesseract-ocr
wget http://users.own-hero.net/~decoder/fuzzyocr/fuzzyocr-3.6.0.tar.gz
tar xvfz fuzzyocr-3.6.0.tar.gz
cd FuzzyOcr-3.6.0/

So FuzzyOCR is now installed, now we need to configure it. FuzzyOCR's configuration file is /etc/spamassassin/FuzzyOcr.cf. In that file almost everything is commented out. We open that file now and make some modifications:

vi /etc/spamassassin/FuzzyOcr.cf

Uncomment the following lines:

focr_global_wordlist /etc/spamassassin/FuzzyOcr.words
focr_preprocessor_file /etc/spamassassin/FuzzyOcr.preps
focr_scanset_file /etc/spamassassin/FuzzyOcr.scansets<
focr_enable_image_hashing 3
focr_digest_db /etc/spamassassin/FuzzyOcr.hashdb
focr_db_hash /etc/spamassassin/FuzzyOcr.db
focr_db_safe /etc/spamassassin/FuzzyOcr.safe.db
focr_bin_helper convert, tesseract

 

Comment out the path:

#focr_path_bin /usr/local/netpbm/bin:/usr/local/bin:/usr/bin

 

We will be storing the image hashes in a mysql database to improve on performance such that images that we have already scanned do not get scanned again as OCR is a resource intense activity.

Create MySQL Database:

The sql script creates the database for fuzzyocr:

mysql -p <  FuzzyOcr.mysql

Grant privileges:

mysql –u root –p
GRANT ALL ON FuzzyOcr.* TO fuzzyocr@localhost IDENTIFIED BY ‘password’;

Update FuzzyOcr.cf

vi /etc/spamassassin/FuzzyOcr.cf

Enable the following lines:
focr_mysql_db FuzzyOcr
focr_mysql_hash Hash
focr_mysql_safe Safe
focr_mysql_user fuzzyocr
focr_mysql_pass fuzzyocr
focr_mysql_host localhost
focr_mysql_port 3306
focr_mysql_socket /var/run/mysqld/mysqld.sock
*Note: Make sure you change the lines in red.

 

Setup FuzzyOcr Database Cleaner

vi /usr/sbin/fuzzy-cleanmysql

#!/usr/bin/perl
#Script to clean out mysql tables of data. Default is to leave data in Safe for 1 day and Hash for 10 days.
#Fuzzyocr-cleanmysql
use Getopt::Long;
use DBI;
use MLDBM qw(DB_File Storable);
my %Files = (
    db_hash => '/var/lib/fuzzyocr/FuzzyOcr.db',
    db_safe => '/var/lib/fuzzyocr/FuzzyOcr.safe.db',
    );
use DBI;
$database = "FuzzyOcr";
$hostname = "localhost";
$socket = "/var/run/mysqld/mysqld.sock";
$port = "3306";
$username = "fuzzyocr";
$password = 'password';
# defaults
my $cfgfile = "/etc/spamassassin/FuzzyOcr.cf";
my %App;
my %age;
$age{'age'} = 10*24;  # 10 days
$age{'hash'} = $age{'age'};
$age{'safe'} = 0;
my $help = 0;
my $verbose = 0;
GetOptions( \%age,
    'age=i',
    'config=s' => \$cfgfile,
    'hash=i',
    'help' => \$help,
    'safe=i',
    'verbose' => \$verbose,
);
if ($help) {
    print "Usage: fuzzy-cleanmysql [Options]\n";
    print "\n";
    print "Available options:\n";
    print "--age=i      Global age in hours to keep in db\n";
    print "--config=s   Specify location of FuzzyOcr.cf\n";
    print "             Default: /etc/spamassassin/FuzzyOcr.cf\n";
    print "--hash=i     Number of hours old to keep in Hash db\n";
    print "--safe=i     Number of hours old to keep in Safe db\n";
    print "--verbose    Show more informations\n";
    print "\n";
    exit 1;
}
# Convert hours to seconds
$age{'age'} *= 60 * 60;
$age{'hash'} *= 60 * 60;
$age{'safe'} *= 60 * 60;
$age{'safe'} = $age{'safe'} ? $age{'safe'} : $age{'age'};
# Read custom paths from FuzzyOcr.cf
my $app_path = q(/usr/local/netpbm/bin:/usr/local/bin:/usr/bin);
open CONFIG, "< $cfgfile" or warn "Can't read configuration file, using defaults...\n";
while () {
    chomp;
    if ($_ =~ m/^focr_bin_(\w+) (.+)/) {
        $App{$1} = $2;
        printf "Found custom path \"$2\" for application \"$1\"\n" if $verbose;
    }
    if ($_ =~ m/^focr_path_bin (.+)/) {
        $app_path = $1;
        printf "Found new path: \"$1\"\n" if $verbose;
    }
    if ($_ =~ m/^focr_enable_image_hashing (\d)/) {
        $App{hashing_type} = $1;
        printf "Found DB Hashing\n" if ($verbose and $1 == 2);
        printf "Found MySQL Hashing\n" if ($verbose and $1 == 3);
    }
    if ($_ =~ m/^focr_mysql_(\w+) (.+)/) {
        $MySQL{$1} = $2;
        printf "Found MySQL option $1 => '$2'\n" if $verbose;
    }
    if ($_ =~ m/^focr_threshold_max_hash (.+)/) {
        $App{max_hash} = $1;
        printf "Updated Thresold{max_hash} = $1\n" if $verbose;
    }
}
close CONFIG;
# make shure we have this threshold set
$App{max_hash} = 5 unless defined $App{max_hash};
# search path for bin_util unless already specified in configuration file
foreach my $app (@bin_utils) {
    next if defined $App{$app};
    foreach my $d (split(':',$app_path)) {
        if (-x "$d/$app") {
            $App{$app} = "$d/$app";
            last;
        }
    }
}
sub get_ddb {
    my %dopts = ( AutoCommit => 1 );
    my $dsn = "DBI:mysql:database=$database";
    if (defined $socket) {
        $dsn .= ";mysql_socket=$socket";
    } else {
        $dsn .= ";host=$hostname";
        $dns .= ";port=$port" unless $port == 3306;
    }
    printf "Connecting to: $dsn\n" if $verbose;
    return DBI->connect($dsn, $username, $password,\%dopts) or die("Could not connect!");
}
if ($App{hashing_type} == 3) {
 my $ddb = get_ddb();
  if ($ddb) {
    my $sql;
    foreach my $ff (sort keys %Files) {
      $ff =~ s/db_//;
      $sqlbase = "FROM $MySQL{$ff} WHERE $MySQL{$ff}.\`check\` < ?";
      my $timestamp = time;
      $timestamp = $timestamp - $age{$ff};
      $sql = "DELETE $sqlbase";
      if ( $verbose ) {
        printf "Delete from Table $MySQL{$ff}\n";
        print "$sql,  $timestamp\n";
        print "Timestamp is ", scalar(localtime($timestamp)), "\n";
        print "That's $age{$ff} seconds earlier than now.\n";
        print "\n";
      }
      $ddb->do($sql,undef,$timestamp);
    }
    $ddb->disconnect;
  }
}

 

chmod +x /usr/sbin/fuzzy-cleanmysql

 

Add it to cron

crontab -e

@weekly /usr/sbin/fuzzy-cleanmysql &> /dev/null  #FuzzyOcr DB cleaner

 

That's it already for the FuzzyOCR configuration. Now let's see if it works as expected.

We can feed each of these emails to SpamAssassin now to see if FuzzyOCR is linked correctly into SpamAssassin.

spamassassin --debug FuzzyOcr < /usr/src/FuzzyOcr-3.6.0/samples/ocr-gif.eml > /dev/null

You should now see a lot of output, the end should look like this:

[...]
[10025] dbg: FuzzyOcr:
[10025] dbg: FuzzyOcr: Friday Augurt 4, 4:01 pm ET
[10025] dbg: FuzzyOcr: LAS VEGAS, NEVADA--(MARKET WIRE)--Aug 4, 2006 -- auantum Energy, lnc. (OTC
[10025] dbg: FuzzyOcr: BB:aEGY.oB-_-
[10025] dbg: FuzzyOcr: auantum Energy, lnc. is pleased to announce that it has applied to have its shares listed for
[10025] dbg: FuzzyOcr: trading on the Frankfurt Stock Exchange. The company has retained the services ofBaltic
[10025] dbg: FuzzyOcr: lnvestment Group of Hamburg, Germany to assist with the application.
[10025] dbg: FuzzyOcr:
[10025] dbg: FuzzyOcr: _ qEGY,OB "

 

12. Filtering PDF, XLS and Phishing Spam with ClamAV (Sanesecurity Signatures)

There is currently a lot of spam where the spam "information" is attached as .pdf or .xls files, sometimes also hidden inside a .zip file. While these spam mails are not easy to catch with e.g. SpamAssassin or a Bayes filter, the ClamAV virus scanner can catch them easily when it is fed with the correct signatures as ClamAV is built to scan mail attachments.

Create a folder for sanesecurity and download and give the script the proper permission.

apt-get install curl rsync
mkdir /usr/src/sanesecurity && cd /usr/src/sanesecurity
wget http://www.inetmsg.com/pub/clamav-unofficial-sigs.tar.gz
tar -zxf clamav-unofficial-sigs.tar.gz && cd clamav-unofficial-sigs-3.7.1
mv clamav-unofficial-sigs.sh /usr/sbin
mv clamav-unofficial-sigs.conf /etc/
chmod +x /usr/sbin/clamav-unofficial-sigs.sh

vi clamav-unofficial-sigs.conf

and change the following variables to match your installation:

clam_dbs="/var/lib/clamav"

The variable clamav_dbs contains the path to the directory where your ClamAV signatures are stored.

Path to clamd.pid:

clamd_pid="/var/run/clamav/clamd.pid"

Reload after update:

reload_dbs="yes"
reload_opt="kill -USR2 `cat $clamd_pid`" #Signals PID to reload dbs

Work Directory:

work_dir="/var/lib/clamav"

And once you're done with the configuration, set the following to yes:

user_configuration_complete="yes" 

Now we run the update script to check if the download works:

clamav-unofficial-sigs.sh

Add it to cron:

00 04 * * * /usr/sbin/clamav-unofficial-sigs.sh -c /etc/clamav-unofficial-sigs.conf &> /dev/null 

Please edit /etc/clamav-unofficial-sigs.conf and comment out the MalwarePortal database section. There has been tons of false positives when using it's defs.

 

13. Greylisting with Greyfix

Greyfix is a tiny greylisting daemon that works extremely well and is easy on resources.

Install:

cd /usr/src && wget http://www.kim-minh.com/pub/greyfix/greyfix-0.3.9.tar.gz
tar -xf greyfix-0.3.9.tar.gz && cd greyfix-0.3.9
./configure --localstatedir=/var
make
make install

vi /etc/postfix/master.cf

and add the following:

greyfix    unix  -        n       n       -        -       spawn
   user=nobody  argv=/usr/local/sbin/greyfix   --greylist-delay 60  -/ 24

We already added the entry for main.cf using the postfix setup script.

Share this page:

39 Comment(s)

Add comment

Comments

From: at: 2010-12-03 02:06:32

Updated.

From: tokamak at: 2010-12-02 01:46:55

 i get an script error for the postfix.sh

Zeile 41: Syntaxfehler beim unerwarteten Wort `(' 

LINE 41: Syntaxerror  unknow word  `('

 

 

 

From: Matt Juaire at: 2011-01-20 01:14:37

I know the howto has this setup on a virtual server. What did you recommend to give to the system for resources (cpu count, memory, hd size)?

From: at: 2011-01-20 15:29:13

This is my current setup:

CPU: 2.8GHZ divided by 3 vms

Ram: 1GB allocated for vm

HDD: 5GB allocated for vm

As you can see, my setup is a vm and I only filter 2 local domains.  Based on your needs, you may need to increase your HDD space for quarantine.  You can install this as a vm or as a stand alone.

From: at: 2011-03-17 16:23:01

your tutorial is very interesting, but i think that it could be more simple for a lot of people to install a solution like  MailCleaner Open Source Edition

Installation CD ISO image and full web admin interface

The result will be quite the same

 Olivier

From: at: 2010-12-21 17:38:17

Hi!

As always, great guide! 

I am using Ubuntu Server 10.04 and in "Install missing perl packages" step I get the following error:

---

root@spamsnake:~# perl -MCPAN -e shell
Terminal does not support AddHistory.

cpan shell -- CPAN exploration and modules installation (v1.9402)
Enter 'h' for help.

cpan[1]> install Crypt::OpenSSL::RSA
CPAN: Storable loaded ok (v2.20)
Going to read '/root/.cpan/Metadata'
  Database was generated on Tue, 21 Dec 2010 16:35:00 GMT
Running install for module 'Crypt::OpenSSL::RSA'
CPAN: Data::Dumper loaded ok (v2.124)
'YAML' not installed, falling back to Data::Dumper and Storable to read prefs '/root/.cpan/prefs'
Running make for I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz
CPAN: Digest::SHA loaded ok (v5.47)
CPAN: Compress::Zlib loaded ok (v2.02)
Checksum for /root/.cpan/sources/authors/id/I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz ok
Scanning cache /root/.cpan/build for sizes
............................................................................DONE
CPAN: Archive::Tar loaded ok (v1.52)
Crypt-OpenSSL-RSA-0.26/
Crypt-OpenSSL-RSA-0.26/RSA.xs
Crypt-OpenSSL-RSA-0.26/RSA.pm
Crypt-OpenSSL-RSA-0.26/typemap
Crypt-OpenSSL-RSA-0.26/MANIFEST
Crypt-OpenSSL-RSA-0.26/Makefile.PL
Crypt-OpenSSL-RSA-0.26/LICENSE
Crypt-OpenSSL-RSA-0.26/Changes
Crypt-OpenSSL-RSA-0.26/t/
Crypt-OpenSSL-RSA-0.26/t/format.t
Crypt-OpenSSL-RSA-0.26/t/bignum.t
Crypt-OpenSSL-RSA-0.26/t/rsa.t
Crypt-OpenSSL-RSA-0.26/README
Crypt-OpenSSL-RSA-0.26/META.yml
CPAN: File::Temp loaded ok (v0.22)

  CPAN.pm: Going to build I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz

Checking if your kit is complete...
Looks good
Warning: prerequisite Crypt::OpenSSL::Random 0 not found.
Writing Makefile for Crypt::OpenSSL::RSA
Could not read '/root/.cpan/build/Crypt-OpenSSL-RSA-0.26-PTQVSZ/META.yml'. Falling back to other methods to determine prerequisites
---- Unsatisfied dependencies detected during ----
----  IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz  ----
    Crypt::OpenSSL::Random [requires]
Shall I follow them and prepend them to the queue
of modules we are processing right now? [yes]
Running make test
  Delayed until after prerequisites
Running make install
  Delayed until after prerequisites
Running install for module 'Crypt::OpenSSL::Random'
'YAML' not installed, falling back to Data::Dumper and Storable to read prefs '/root/.cpan/prefs'
Running make for I/IR/IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz
Checksum for /root/.cpan/sources/authors/id/I/IR/IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz ok
Crypt-OpenSSL-Random-0.04/
Crypt-OpenSSL-Random-0.04/Random.pm
Crypt-OpenSSL-Random-0.04/Random.xs
Crypt-OpenSSL-Random-0.04/LICENSE
Crypt-OpenSSL-Random-0.04/Changes
Crypt-OpenSSL-Random-0.04/test.pl
Crypt-OpenSSL-Random-0.04/Makefile.PL
Crypt-OpenSSL-Random-0.04/META.yml
Crypt-OpenSSL-Random-0.04/MANIFEST

  CPAN.pm: Going to build I/IR/IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz

Checking if your kit is complete...
Looks good
Writing Makefile for Crypt::OpenSSL::Random
Could not read '/root/.cpan/build/Crypt-OpenSSL-Random-0.04-T4RbJx/META.yml'. Falling back to other methods to determine prerequisites
cp Random.pm blib/lib/Crypt/OpenSSL/Random.pm
AutoSplitting blib/lib/Crypt/OpenSSL/Random.pm (blib/lib/auto/Crypt/OpenSSL/Random)
/usr/bin/perl /usr/share/perl/5.10/ExtUtils/xsubpp  -typemap /usr/share/perl/5.10/ExtUtils/typemap  Random.xs > Random.xsc && mv Random.xsc Random.c
Please specify prototyping behavior for Random.xs (see perlxs manual)
cc -c   -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g   -DVERSION=\"0.04\" -DXS_VERSION=\"0.04\" -fPIC "-I/usr/lib/perl/5.10/CORE"   Random.c
Random.xs:5:26: error: openssl/rand.h: No such file or directory
make: *** [Random.o] Error 1
  IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz
  /usr/bin/make -- NOT OK
Warning (usually harmless): 'YAML' not installed, will not store persistent state
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
Running make for I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz
  Has already been unwrapped into directory /root/.cpan/build/Crypt-OpenSSL-RSA-0.26-PTQVSZ

  CPAN.pm: Going to build I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz

Warning: Prerequisite 'Crypt::OpenSSL::Random => 0' for 'IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz' failed when processing 'IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz' with 'make => NO'. Continuing, but chances to succeed are limited.
CPAN: Time::HiRes loaded ok (v1.9719)
cp RSA.pm blib/lib/Crypt/OpenSSL/RSA.pm
AutoSplitting blib/lib/Crypt/OpenSSL/RSA.pm (blib/lib/auto/Crypt/OpenSSL/RSA)
/usr/bin/perl /usr/share/perl/5.10/ExtUtils/xsubpp  -typemap /usr/share/perl/5.10/ExtUtils/typemap -typemap typemap  RSA.xs > RSA.xsc && mv RSA.xsc RSA.c
cc -c   -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g   -DVERSION=\"0.26\" -DXS_VERSION=\"0.26\" -fPIC "-I/usr/lib/perl/5.10/CORE"  -DPERL5 -DOPENSSL_NO_KRB5 RSA.c
RSA.xs:5:25: error: openssl/bio.h: No such file or directory
[several errors like above...]
make: *** [RSA.o] Error 1
  IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz
  /usr/bin/make -- NOT OK
Warning (usually harmless): 'YAML' not installed, will not store persistent state
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
Failed during this command:
 IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz    : make NO
 IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz       : make NO

cpan[2]>

---

Any ideas of how to fix it?

From: at: 2010-12-23 06:51:38

You do not have the openssl header files. Random.xs:5:26: error: openssl/rand.h: No such file or directory

 Install the openssl-dev package

From: at: 2012-03-12 00:21:52

on ubuntu its

apt-get install libssl-dev

regards

From: Naz at: 2010-12-08 03:48:32

Hi, thank you for the great how-to. pyzor_add_header 1 is no longer a valid config option with the newer versions of spamassassin and will generate warning. You can see it at the --lint output.

From: at: 2010-12-08 15:22:56

Updated thanks.

From: Anonymous at: 2010-12-07 15:07:58

 spamassing test error
 
Dec  7 15:20:15.262 [15606] warn: config: failed to parse line, skipping, in "/etc/spamassassin/mailscanner.cf": pyzor_add_header 1
Dec  7 15:20:15.264 [15606] warn: config: failed to parse line, skipping, in "/opt/MailScanner/etc/spam.assassin.prefs.conf": pyzor_add_header 1                                          
best regards
 
 
 

From: at: 2010-12-04 17:11:04

Guide updated, mailscanner should be installed first.

From: Anonymous at: 2010-12-04 09:51:46

there is no directory: cp /opt/MailScanner/etc/spam.assassin.prefs.conf

 

From: Eddo at: 2010-11-30 14:49:36

Great I was waiting for this one!

At step 7 I think you mean the libclamav-client-perl?

Regards,

From: Eddo at: 2010-11-30 14:58:36

Great I was waiting for this one!

At step 7 I think you mean the libclamav-client-perl? and should we install spamassassin here or download it and install from source?

Regards,

From: at: 2010-12-01 19:16:11

It should be libclamav6. Also, I moved the dependencies up a bit, we want it before the spamassassin section.


From: w0rldart at: 2011-01-17 13:03:42

Hi, i am suposed to look for 

debug: bayes: Database connection established
debug: bayes: found bayes db version 3
debug: bayes: Using userid: 2

as response to  spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf --lint , but in stead i get

 Jan 17 13:56:55.129 [10360] dbg: timing: total 1152 ms - init: 770 (66.8%), parse: 0.81 (0.1%), extract_message_metadata: 1.36 (0.1%), get_uri_detail_list: 0.98 (0.1%), tests_pri_-1000: 7 (0.6%), compile_gen: 149 (12.9%), compile_eval: 16 (1.4%), tests_pri_-950: 5 (0.4%), tests_pri_-900: 5 (0.5%), tests_pri_-400: 5 (0.4%), tests_pri_0: 309 (26.8%), tests_pri_500: 45 (3.9%)

Jan 17 13:56:55.129 [10360] warn: lint: 2 issues detected, please rerun with debug enabled for more information

 

 Can any1 help me out?

From: at: 2011-01-18 13:40:28

Hi,

Please post your issue in the support forum and we'll gladly help you out.

Thanks,

Rocky

From: Anvar at: 2011-02-10 07:58:09

Maybe handy to add the location of the master.cf file; /etc/postfix

From: Anvar at: 2011-02-10 08:30:57

Best to install the clamav data;

 apt-get install clamav-data and afterwards /etc/init.d/clamav-daemon start

From: Alexander Meesters at: 2011-04-06 12:33:52

i think its better to use:

 sudo update-rc.d mailscanner defaults

 then creating it by hand...

From: lugi at: 2011-04-15 16:05:24

When i trying to lauch => install Crypt::OpenSSL::RSA i have this problem . Can you help me please.

I trying => Crypt::OpenSSL::Random  but it doesn't work also 

 

Checking if your kit is complete...
Looks good
Writing Makefile for Crypt::OpenSSL::Random
cp Random.pm blib/lib/Crypt/OpenSSL/Random.pm
AutoSplitting blib/lib/Crypt/OpenSSL/Random.pm (blib/lib/auto/Crypt/OpenSSL/Random)
/usr/bin/perl /usr/share/perl/5.10/ExtUtils/xsubpp  -typemap /usr/share/perl/5.10/ExtUtils/typemap  Random.xs > Random.xsc && mv Random.xsc Random.c
Please specify prototyping behavior for Random.xs (see perlxs manual)
cc -c   -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g   -DVERSION=\"0.04\" -DXS_VERSION=\"0.04\" -fPIC "-I/usr/lib/perl/5.10/CORE"   Random.c
Random.xs:5: fatal error: openssl/rand.h: No such file or directory
compilation terminated.
make: *** [Random.o] Error 1
  IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz
  /usr/bin/make -- NOT OK
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
Running make for I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz
  Has already been unwrapped into directory /root/.cpan/build/Crypt-OpenSSL-RSA-0.26-OaSkf7

  CPAN.pm: Going to build I/IR/IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz

Warning: Prerequisite 'Crypt::OpenSSL::Random => 0' for 'IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz' failed when processing 'IROBERTS/Crypt-OpenSSL-Random-0.04.tar.gz' with 'make => NO'. Continuing, but chances to succeed are limited.
CPAN: Time::HiRes loaded ok (v1.9719)
cp RSA.pm blib/lib/Crypt/OpenSSL/RSA.pm
AutoSplitting blib/lib/Crypt/OpenSSL/RSA.pm (blib/lib/auto/Crypt/OpenSSL/RSA)
/usr/bin/perl /usr/share/perl/5.10/ExtUtils/xsubpp  -typemap /usr/share/perl/5.10/ExtUtils/typemap -typemap typemap  RSA.xs > RSA.xsc && mv RSA.xsc RSA.c
cc -c   -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g   -DVERSION=\"0.26\" -DXS_VERSION=\"0.26\" -fPIC "-I/usr/lib/perl/5.10/CORE"  -DPERL5 -DOPENSSL_NO_KRB5 RSA.c
RSA.xs:5: fatal error: openssl/bio.h: No such file or directory
compilation terminated.
make: *** [RSA.o] Error 1
  IROBERTS/Crypt-OpenSSL-RSA-0.26.tar.gz
  /usr/bin/make -- NOT OK
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible

From: at: 2011-04-18 17:53:20

Try:

apt-get install libcrypt-openssl-random-perl libcrypt-openssl-rsa-perl

From: Tony Grenda at: 2011-05-12 19:44:07

I had to open Port 873/TCP on my firewall for the rsync protocol to work for the SaneSecurity signatures to download.

From: at: 2011-12-06 19:43:45

How can i customize spamassassin rules? baruwa is reporting valid mail marked as spam:

3.09  DOS_OE_TO_MX                  Delivered direct to MX with OE headers
0.00  DYN_RDNS_SHORT_HELO_HTML      Sent by dynamic rDNS, short HELO, and HTML
0.00  FSL_HELO_NON_FQDN_1
0.00  HELO_NO_DOMAIN                Relay reports its domain incorrectly
0.00  HTML_MESSAGE                  HTML included in message
3.56  RCVD_IN_PBL                   Received via a relay in Spamhaus PBL
1.28  RCVD_IN_RP_RNBL               Relay in RNBL, https://senderscore.org/blacklistlookup/
0.36  RDNS_DYNAMIC                  Delivered to internal network by host with dynamic-looking rDNS
Thanks

From: Eddo Jansen at: 2010-12-09 12:53:50

Even after your latest changes I get the Insecure dependency error.

save_execute: Insecure dependency in open while running with -T switch at /usr/share/perl5/FuzzyOcr/Misc.pm line 92.
save_execute: Insecure dependency in open while running with -T switch at /usr/share/perl5/FuzzyOcr/Misc.pm line 92.
Dec  9 13:49:16.320 [26635] dbg: FuzzyOcr: Elapsed [26653]: 0.041086 sec. (/usr/bin/giftext: exit 8)
Dec  9 13:49:16.320 [26635] warn: readline() on closed filehandle INFILE at /usr/share/perl5/FuzzyOcr/Misc.pm line 205.
Dec  9 13:49:16.321 [26635] info: FuzzyOcr: Image is single non-interlaced...
Dec  9 13:49:16.324 [26635] warn: rules: failed to run FUZZY_OCR test, skipping:
Dec  9 13:49:16.324 [26635] warn:  (Insecure dependency in printf while running with -T switch at /usr/share/perl5/FuzzyOcr.pm line 469.
Dec  9 13:49:16.324 [26635] warn: )
root@srvnld0005:/usr/src# Insecure dependency in printf while running with -T switch at /usr/share/perl5/FuzzyOcr.pm line 469.

apt-get install fuzzyocr 3 does not work for me... only if I do not configure the database...

Any thoughts on this?

From: at: 2010-12-10 22:13:37

For future problems, please post support questions in the forum.

 Please remove Fuzzy by doing:

apt-get install fuzzyocr netpbm gifsicle libungif-bin gocr ocrad libstring-approx-perl libmldbm-sync-perl libdigest-md5-perl libdbd-mysql-perl imagemagick tesseract-ocr fuzzyocr3

If anything is left, run apt-get autoremove to get rid of them.

Redo the complete FuzzyOcr section and you shouldn't have any problems.

From: at: 2010-12-04 20:34:29

Hmm, that is really strange, I must have uploaded the wrong baruwa to my storage.  I have a modified version for this install. 

 Link is good now.

From: Eddo Jansen at: 2010-12-03 12:34:31

Great guide but I came across some minor issue's...

Beside making the Spamassassin modifications after you install MailScanner, Baruwa installs a package maintained version of MailScanner as a missing dependency and places the essential .pm files in /etc/MailScanner/CustomFunctions...

You will have to move these files to the correct location:

mv /etc/MailScanner/CustomFunctions/Baruwa* /opt/MailScanner/lib/MailScanner/CustomFunctions/

Rename the old version of MailScanner to avoid problems:

mv /etc/MailScanner/ /etc/MailScanner_obsolete

Make sure you have not overwritten the /etc/init.d/mailscanner script by installing the Baruwa dependencies, otherwise make the appropriate changes to that file.

I thought I should share this with you, it might give people a headache (It gave me one  )

Cheers!

From: Sander de Rijk at: 2011-02-11 07:15:19

Great guide, I followed it and it indeed stops all my spam with no false positives :)

A few notes though: 

Baruwa recommends to install it on apache and most people already have apache running. It took me some extra work to get it running on apache but I suggest you include that in your guide or a link telling that you can actually also run it on apache.

The second one is SPF related. My DNS service also has a relayservice for email in case my mailserver is down. The current SPF setup will bounce mail because it doesnt understand that the relayservice should be trusted for the SPF module. I turned it off and still need to dig into it so that host is excluded from SPF checks.

 If you install all the cronjobs that you specified directly you will get ALOT of emails from the mailserver. I suggest to test them and then add &> /dev/null at the end of every line.

From: at: 2011-02-15 02:03:46

Thank you, really appreciate your suggestions.

Baruwa can be installed on any webserver than can run Django and nginx happens to be very fast and light.  Apache is supported out of the box with the distributed release but my custom package is strictly for nginx, which was chosen to make the system as slim as possible.

As for the SPF issue, this build has a global whitelist, which will bypass all checks once you have that ip/host in the whitelist.  Go ahead and whitelist your relayhost and that should bypass SPF checks.

From: John M at: 2011-04-06 18:40:26

The /usr/sbin/fuzzy-cleanmysql didn't work for my until I changed the following line while () { to while(<CONFIG>) { A nasty infinite loop occurs otherwise.

From: at: 2011-04-20 13:49:19

Thanks.

From: Tony Grenda at: 2011-05-12 18:49:28

I had to create a symlink to the /tmp/mysql.sock file since Ubuntu does not use this file (it is /var/run/mysqld/mysqld.sock). Use the next line to creat the symlink. sudo ln -s /var/run/mysqld/mysqld.sock /tmp/mysql.sock I could not get the FuzzyOcr check using spamassassin to work properly until I made the change.

From: at: 2011-06-10 08:08:56

Upgrade worked like a charm, thanks.

From: at: 2011-07-12 07:13:00

Great guide but shouldnt you have Baruwa installed AFTER Nginx? as you call for it to be restarted in step 8 but it isnt installed until step 9.

From: at: 2012-02-16 13:18:01

The confusion is caused by the upgrade section.  You have to completely finish the build first then go back and do the upgrade.

From: kup at: 2011-10-24 08:25:47

Hello. For the first, I would like to say - this is a great howto. My question ... do you have in your repositories the latest version of Baruwa frontend (1.1.1)? Many thanks.

From: at: 2012-02-16 13:18:38

Done, check the guide.