13 Filtering PDF, XLS and Phishing Spam with ClamAV (Sanesecurity Signatures)

There is currently a lot of spam where the spam "information" is attached as .pdf or .xls files, sometimes also hidden inside a .zip file. While these spam mails are not easy to catch with e.g. SpamAssassin or a Bayes filter, the ClamAV virus scanner can catch them easily when it is fed with the correct signatures as ClamAV is built to scan mail attachments.

13.1 Create a Folder and Download the Script

Create a folder for sanesecurity and download and give the script the proper permission.

apt-get install curl

mkdir /usr/src/sanesecurity
cd /usr/src/sanesecurity
wget http://www.sanesecurity.co.uk/clamav/ss-msrbl.txt
mv ss-msrbl.txt /usr/bin/ss_update.sh
chmod +x /usr/bin/ss_update.sh

Edit ss_update.sh and change the following variables to match your installation:

clam_sigs="/var/lib/clamav"

The variable clamav_sigs contains the path to the directory where your ClamAV signatures are stored.

clam_user="clamav"

Now we run the update script to check if the download works:

./ss_update.sh

The result should look similar to this:

=================================
SaneSecurity SCAM Database Update
=================================
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 116k 100 116k 0 0 65448 0 0:00:01 0:00:01 --:--:-- 139k

==================================
SaneSecurity PHISH Database Update
==================================
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 179k 100 179k 0 0 216k 0 --:--:-- --:--:-- --:--:-- 216k

==========================
MSRBL SPAM Database Update
==========================
Number of files: 1
Number of files transferred: 1
Total file size: 228436 bytes
Total transferred file size: 228436 bytes
Literal data: 228436 bytes
Matched data: 0 bytes
File list size: 33
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 101
Total bytes received: 228579
sent 101 bytes received 228579 bytes 26903.53 bytes/sec
total size is 228436 speedup is 1.00

===========================
MSRBL IMAGE Database Update
===========================
Number of files: 1
Number of files transferred: 1
Total file size: 550503 bytes
Total transferred file size: 550503 bytes
Literal data: 550503 bytes
Matched data: 0 bytes
File list size: 35
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 103
Total bytes received: 550688
sent 103 bytes received 550688 bytes 157368.86 bytes/sec
total size is 550503 speedup is 1.00

Now we a add the script to the root crontab to be run once a day:

crontab -e

Add the following line at the end of the root crontab:

00 04 * * * /usr/bin/ss_update.sh &> /dev/null

14 GreyListing with Postfix-gld

14.1 Installing Postfix-gld

apt-get install postfix-gld

Create MySQL Database:

mysql –u root –p

mysql> create database gld_db
mysql> GRANT ALL ON gld_db.* TO gld_user@localhost IDENTIFIED BY ‘gld_password’;
mysql> flush privileges;

Import tables.mysql:

mysql –u gld_user –p gld_db < /usr/share/gld/tables.mysql

Import table-whitelist.sql:

mysql –u gld_user –p gld_db < /usr/share/gld/table-whitelist.sql

You will have to enable it by configuring that in the /etc/default/gld:

vi /etc/default/gld

#/etc/default/gld
ENABLED=1

14.2 Configuration

Edit /etc/gld.conf according to your needs. I'm using the following settings:

vi /etc/gld.conf

# Config file for gld
# TCP Port gld should listen to (default is 2525)
#
PORT=2525
# Shall we bind only to loopback ? (0=No,1=Yes) (default is 1)
LOOPBACKONLY=1
# The list of networks allowed to connect to us (default is everybody)
CLIENTS=127.0.0.1/32
# The user used to run gld (default value is no user change)
USER=postfix-gld
# The group used to run gld (default value is no group change)
GROUP=postfix-gld
# Maximum simultaneous connexions (default is 100)
MAXCON=100
# How many seconds we should wait before accepting a mail that is in greylist (default is 60)
MINTIME=60
# Shall we use lightgrey option ? (0=No,1=Yes) (default is 0)
# The lightgrey option, mask the last octet of IP addresses
# and thus we greylist only C classes (/24) instead of individual IPs.
LIGHTGREY=0
# Shall we use the mxgrey algorithm ? (0=No,>0=Yes) (default is 0)
# the mxgrey algorithm is a variation of the greylist algorithm.
# When this is enabled, we allow all incoming mails from an IP address
# whatever source/destination email as long as this IP has been greylisted
# at least X time and succeded the mail resend .
#
# Example:
# The IP 1.2.3.4 sends an email from src@domain.com to user@yourdomain.com
# We greylist this mail as this IP is not yet in database and send a 450 SMTP code
# After some time, the IP re-send the mail from src@domain.com to user@yourdomain.com
# We update the db.
# Some time after the ip 1.2.3.4 sends an email from john@domain.com to fred@yourdomain.com
# We will accept this mail without any greylisting, as this ip already succeded a greylist test
# and thus seems to be a valid smtp server and not a spammer .
#
# The advantage of this method, is that it reduce the re-send time due to greylisting to
# x mail per server instead of one mail per destination .
#
# The value you provide in MXGREY is the minimum number of succesful greylists
# before accepting all mails from this MX. higher the number is, harder is to get in.
#
# This algortihm replace the old LIGHTGREYDOMAIN which was available prior version 1.6
#
MXGREY=1
# Shall we use the whitelist table ?  (0=No,1=Yes) (default is 1)
# If set to yes, then the table 'whitelist' is looked up
# each time postfix request the server
# if the email/domain/ip is in the whitelist, then the response
# will be 'dunno' .
# In the whitelist table, you can set the following values:
# an email: ie john@foo.tld
# a domain: ie @bar.tld
# an IP   : ie 1.2.3.4
# a subnet: ie 1.2.3
#
WHITELIST=1
# Shall we use a DNS based whitelist ? (default is no)
# To activate it, the line must be uncommented
# and the value set to the domain of the DNS whitelist.
# for example, if DNSWL is set to toto.com and we get a mail from ip a.b.c.d
# then gld will DNS lookup d.c.b.a.toto.com
# and if found allow the ip without greylisting it.
#DNSWL=toto.com
# Shall we send a 'dunno' in case of error (mysql down,....) (0=No,1=Yes) (default is 1)
# Normaly, if an error occur, the server is supposed to close the connection
# and thus postfix will return a 450 Server configuration error
# if this parameter is set to 1, then the server will return 'dunno'
# and thus let postfix decide the fate of the mail.
ERRACCEPT=1
# Shall we log to the syslog (0=No,1=Yes) (default is 1)
SYSLOG=1
# If we use syslog, which facility shall we use (default is mail)
# it can only be one of the following facilities:
# daemon mail local0 local1 local2 local3 local4 local5 local6 local7
FACILITY=mail
# The Message that we display in case of reject (default is "Greylisted")
# If you want another SMTP return code than the default 450, just put it at
# the beginning of the message, ie: 451 You have been greylisted by gld ...
# If you don't provide any SMTP code, the default 450 will be used by postfix
# WARNING: if you set a custom smtp code make sure it's a 4XX code.
# if you don't provide a 4XX code, gld will ignore it and send the default 450.
# Be also warned that if you set a custom code, gld will not use defer_if_permit anymore
# but direct supplied code to postfix .
MESSAGE=Service temporarily unavailable, please try later
# Training mode activated ? (0=No,1=Yes) (default is 0)
# If activated, gld will do all the work but will always reply dunno to postfix
# and thus, will never greylist any mail.
# This feature is useful for testing gld performances without greylisting any mail
TRAINING=0
# SQL INFOS (defaults are localhost,myuser,mypasswd,mydb)
#
SQLHOST=localhost
SQLUSER=gld_user
SQLPASSWD=gld_password
SQLDB=gld_db

Edit /etc/postfix/main.cf and add the following to smtpd_recipient_restrictions:

vi /etc/postfix/main.cf

check_policy_service inet:127.0.0.1:2525

Do a

tail –f /var/log/mail.log

and check your log for the following:

Apr 28 09:07:03 server1 gld: Greylist activated for recipient=<xxx@xxx.com> sender=<xxx@xxx.com> ip=<xxx.xxx.xxx.xxx>

You can set up a cron job to keep your database clean. Below is what I'm using in crontab -e running at midnight.

@daily /usr/bin/mysql -ugld_user -pgld_password -e 'USE gld_db; DELETE FROM greylist WHERE n > 0;' &> /dev/null

15 Logwatch Statistical Reporting (Optional)

Logwatch is a customizable log analysis system. Logwatch parses through your system's logs for a given period of time and creates a report analyzing areas that you specify, in as much detail as you require.

We will be using Logwatch to give us daily reports for mailscanner. This is a way for us to see how effective mailscanner really is.

Install Logwatch:

apt-get install logwatch

Edit the /usr/share/logwatch/default.conf/logwatch.conf and set the options:

vi /usr/share/logwatch/default.conf/logwatch

Mail To = youremailaddress
Service = mailscanner

Test Logwatch:

logwatch

It should generate a log file and email it to the email you specified.

16 Automatically Add A Disclaimer To Outgoing Emails With alterMIME (Optional)

This tutorial shows how to install and use alterMIME. alterMIME is a tool that can automatically add a disclaimer to emails. In this article I will explain how to install it as a Postfix filter on Ubuntu.

16.1 Installing alterMIME

alterMIME can be installed as follows:

apt-get install altermime

Next we create the user filter with the home directory /var/spool/filter - alterMIME will be run as that user:

useradd -r -c "Postfix Filters" -d /var/spool/filter filter
mkdir /var/spool/filter
chown filter:filter /var/spool/filter
chmod 750 /var/spool/filter

Afterwards we create the script /etc/postfix/disclaimer which executes alterMIME. Ubuntu's alterMIME package comes with a sample script that we can simply copy to /etc/postfix/disclaimer:

cp /usr/share/doc/altermime/examples/postfix_filter.sh /etc/postfix/disclaimer
chgrp filter /etc/postfix/disclaimer
chmod 750 /etc/postfix/disclaimer

Now the problem with this script is that it doesn't distinguish between incoming and outgoing emails - it simply adds a disclaimer to all mails. Typically you want disclaimers only for outgoing emails, and even then not for all sender addresses. Therefore I've modified the /etc/postfix/disclaimer script a little bit - we'll come to that in a minute.

Right now, we create the file /etc/postfix/disclaimer_addresses which holds all sender email addresses (one per line) for which alterMIME should add a disclaimer:

vi /etc/postfix/disclaimer_addresses

user1@example.com
user2@example.org
user3@example.net

Now we open /etc/postfix/disclaimer and modify it as follows (I have marked the parts that I've changed):

vi /etc/postfix/disclaimer

#!/bin/sh
# Localize these.
INSPECT_DIR=/var/spool/filter
SENDMAIL=/usr/sbin/sendmail
####### Changed From Original Script #######
DISCLAIMER_ADDRESSES=/etc/postfix/disclaimer_addresses
####### Changed From Original Script END #######
# Exit codes from <sysexits.h>
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
# Clean up when done or when aborting.
trap "rm -f in.$$" 0 1 2 3 15
# Start processing.
cd $INSPECT_DIR || { echo $INSPECT_DIR does not exist; exit
$EX_TEMPFAIL; }
cat >in.$$ || { echo Cannot save mail to file; exit $EX_TEMPFAIL; }
####### Changed From Original Script #######
# obtain From address
from_address=`grep -m 1 "From:" in.$$ | cut -d "<" -f 2 | cut -d ">" -f 1`
if [ `grep -wi ^${from_address}$ ${DISCLAIMER_ADDRESSES}` ]; then
  /usr/bin/altermime --input=in.$$ \
                   --disclaimer=/etc/postfix/disclaimer.txt \
                   --disclaimer-html=/etc/postfix/disclaimer.txt \
                   --xheader="X-Copyrighted-Material: Please visit http://www.company.com/privacy.htm" || \
                    { echo Message content rejected; exit $EX_UNAVAILABLE; }
fi
####### Changed From Original Script END #######
$SENDMAIL "$@" <in.$$
exit $?

Next we need the text file /etc/postfix/disclaimer.txt which holds our disclaimer text. Ubuntu's alterMIME package comes with a sample text that we can use for now (of course, you can modify it if you like):

cp /usr/share/doc/altermime/examples/disclaimer.txt /etc/postfix/disclaimer.txt

Finally we have to tell Postfix that it should use the /etc/postfix/disclaimer script to add disclaimers to outgoing emails. Open /etc/postfix/master.cf and add -o content_filter=dfilt: to the smtp line:

vi /etc/postfix/master.cf

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
   -o content_filter=dfilt:
[...]

At the end of the same file, add the following two lines:

[...]
dfilt     unix    -       n       n       -       -       pipe
    flags=Rq user=filter argv=/etc/postfix/disclaimer -f ${sender} -- ${recipient}

Restart Postfix afterwards:

/etc/init.d/postfix restart

That's it! Now a disclaimer should be added to outgoing emails sent from the addresses listed in /etc/postfix/disclaimer_addresses.

17 Firewalling the SpamSnake with Firehol

Introduction

Firehol is a stateful iptables packet filtering firewall configurator. It is abstracted, extensible, easy and powerful. It can handle any kind of firewall, but most importantly, it gives you the means to configure it, the same way you think of it.

17.1 Install Firehol

Install firehol by doing:

apt-get install firehol

17.2 Firehol Settings

Edit /etc/default/firehol and change the following:

vi /etc/default/firehol

START_FIREHOL=NO

to

START_FIREHOL=YES

Edit /etc/firehol/firehol.conf and add the following:

vi /etc/firehol/firehol.conf

version 5
# Accept all client traffic on any interface
interface any internet
        protection strong
        server "icmp ping ICMP ssh http https telnet samba webmin dns dcc echo smtp" accept
        client all accept

Be sure to comment out the default configuration before applying these settings.  This filters all incoming connections that are not related to the above services.

If you want to be less polite, you can drop them by adding the following after 'protection strong': policy drop

17.3 Updating RESERVED_IPS list

cd /usr/src
wget http://firehol.sf.net/firehol.tar.gz
tar xzvf firehol.tar.gz
cd firehol
mv get-iana.sh /usr/bin/get-iana.sh
chmod +x /usr/bin/get-iana.sh

Run the script to update RESERVED_IPS:

get-iana.sh

Make sure to select 'yes' when asked if you would like to save RESERVED_IPS to /etc/firehol/RESERVED_IPS.

Start Firehol:

/etc/init.d/firehol start

Now your server is set up to only accept connections for the services you allowed.

Congratulations!

You should now have a complete working SpamSnake.

Here are some Mailwatch screenshots: