The Perfect SpamSnake - Ubuntu Jeos 9.10 - Page 4
8. Install and Configure SPF
The postfix-policyd-spf-perl package depends on the Mail::SPF and the NetAddr::IP Perl modules.
We need to download postfix-policyd-spf-perl from http://www.openspf.org/Software to the /usr/src/ directory and install it to the /usr/lib/postfix/ directory like this:
cd /usr/src
wget http://www.openspf.org/blobs/postfix-policyd-spf-perl-2.007.tar.gz
tar xvfz postfix-policyd-spf-perl-2.007.tar.gz
cd postfix-policyd-spf-perl-2.007
cp postfix-policyd-spf-perl /usr/lib/postfix/policyd-spf-perl
Then we edit /etc/postfix/master.cf and add the following stanza at the end:
vi /etc/postfix/master.cf
policy unix - n n - - spawn user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl
(The leading spaces before user=nobody are important so that Postfix knows that this line belongs to the previous one!)
Then open /etc/postfix/main.cf and search for the smtpd_recipient_restrictions directive. You should have reject_unauth_destination in that directive, and right after reject_unauth_destination you add check_policy_service unix:private/policy like this:
vi /etc/postfix/main.cf
[...] smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination,check_policy_service unix:private/policy [...]
or like this:
[...] smtpd_recipient_restrictions = reject_unauth_destination,check_policy_service unix:private/policy [...]
It is important that you specify check_policy_service AFTER reject_unauth_destination or else your system can become an open relay!
Then restart Postfix:
/etc/init.d/postfix restart
That's it already.
9. Install and Configure FuzzyOcr
apt-get install netpbm gifsicle libungif-bin gocr ocrad libstring-approx-perl libmldbm-sync-perl imagemagick tesseract-ocr
Download and install the latest FuzzyOCR devel version from http://fuzzyocr.own-hero.net/wiki/Downloads:
cd /usr/src/
wget http://users.own-hero.net/~decoder/fuzzyocr/fuzzyocr-3.5.1-devel.tar.gz
Unpack FuzzyOCR and move all FuzzyOcr* files and the FuzzyOcr directory (they are all in the FuzzyOcr-3.5.1/ directory) to /etc/mail/spamassassin:
tar xvfz fuzzyocr-3.5.1-devel.tar.gz
cd FuzzyOcr-3.5.1/
mv FuzzyOcr* /etc/mail/spamassassin/
wget http://www.gbnetwork.co.uk/mailscanner/FuzzyOcr.words -O /etc/mail/spamassassin/FuzzyOcr.words
We will be storing the image hashes in a mysql database to improve on performance such that images that we have already scanned do not get scanned again as OCR is a resource intense activity.
Create MySQL Database
:
mysql -p < /etc/mail/spamassassin/FuzzyOcr.mysql
Setup the password:
mysql -u root -p
mysql> GRANT ALL ON FuzzyOcr.* TO fuzzyocr@localhost IDENTIFIED BY 'password';
MailWatch Fix
Do the following to prevent an error in MailWatch:
vi /etc/mail/spamassassin/FuzzyOcr.pm
Change
'use POSIX;'
to
'use POSIX qw(SIGTERM);'
FuzzyOcr Configuration
FuzzyOCR's configuration file is /etc/mail/spamassassin/FuzzyOcr.cf. In that file almost everything is commented out. We open that file now and make some modifications:
vi /etc/mail/spamassassin/FuzzyOcr.cf
Put the following line into it to define the location of FuzzyOCR's spam words file:
focr_global_wordlist /etc/mail/spamassassin/FuzzyOcr.words
/etc/mail/spamassassin/FuzzyOcr.words is a predefined word list that comes with FuzzyOCR. You can adjust it to your needs. Next change:
# Include additional scanner/preprocessor commands here: # focr_bin_helper pnmnorm, pnminvert, pamthreshold, ppmtopgm, pamtopnm focr_bin_helper tesseract
to
# Include additional scanner/preprocessor commands here: # focr_bin_helper pnmnorm, pnminvert, convert, ppmtopgm, tesseract
Finally add/enable the following lines:
# Search path for locating helper applications focr_path_bin /usr/local/netpbm/bin:/usr/local/bin:/usr/bin focr_preprocessor_file /etc/mail/spamassassin/FuzzyOcr.preps focr_scanset_file /etc/mail/spamassassin/FuzzyOcr.scansets focr_digest_db /etc/mail/spamassassin/FuzzyOcr.hashdb focr_db_hash /etc/mail/spamassassin/FuzzyOcr.db focr_db_safe /etc/mail/spamassassin/FuzzyOcr.safe.db focr_minimal_scanset 1 focr_autosort_scanset 1 focr_enable_image_hashing 3 focr_logfile /var/log/FuzzyOcr.log #Mysql Connection# focr_mysql_db FuzzyOcr focr_mysql_hash Hash focr_mysql_safe Safe focr_mysql_user fuzzyocr focr_mysql_pass password focr_mysql_host localhost focr_mysql_port 3306 focr_mysql_socket /var/run/mysqld/mysqld.sock
This is what the FuzzyOCR developers say about image hashing:
"The Image hashing database feature allows the plugin to store a vector of image features to a database, so it knows this image when it arrives a second time (and therefore does not need to scan it again). The special thing about this function is that it also recognizes the image again if it was changed slightly (which is done by spammers)."
Test FuzzyOCR
cd /usr/src/FuzzyOcr-3.5.1/samples
spamassassin --debug FuzzyOcr < ocr-animated.eml > /dev/null
You see the following:
[14808] info: FuzzyOcr: Found Score <9.000> for Exact Image Hash
[14808] info: FuzzyOcr: Matched [1] time(s). Prev match: 16 sec. ago
[14808] info: FuzzyOcr: Message is SPAM. Words found:
[14808] info: FuzzyOcr: "price" in 1 lines
[14808] info: FuzzyOcr: "company" in 1 lines
[14808] info: FuzzyOcr: "alert" in 1 lines
[14808] info: FuzzyOcr: "news" in 1 lines
[14808] info: FuzzyOcr: (6 word occurrences found)
[14808] dbg: FuzzyOcr: Remove DIR: /tmp/.spamassassin14808JZSvHBtmp
[14808] dbg: FuzzyOcr: Processed in 0.104555 sec.
10. Filtering PDF, XLS and Phishing Spam with ClamAV (Sanesecurity Signatures)
There is currently a lot of spam where the spam "information" is attached as .pdf or .xls files, sometimes also hidden inside a .zip file. While these spam mails are not easy to catch with e.g. SpamAssassin or a Bayes filter, the ClamAV virus scanner can catch them easily when it is fed with the correct signatures as ClamAV is built to scan mail attachments.
Create a folder for sanesecurity and download and give the script the proper permission.
apt-get install curl rsync
mkdir /usr/src/sanesecurity && cd /usr/src/sanesecurity
wget http://www.inetmsg.com/pub/clamav-unofficial-sigs.tar.gz
tar -zxf clamav-unofficial-sigs.tar.gz && cd clamav-unofficial-sigs-3.7
mv clamav-unofficial-sigs.sh /usr/bin
mv clamav-unofficial-sigs.conf /etc/
chmod +x /usr/bin/clamav-unofficial-sigs.sh
Edit clamav-unofficial-sigs.conf and change the following variables to match your installation:
clam_dbs="/var/lib/clamav"
The variable clamav_dbs contains the path to the directory where your ClamAV signatures are stored.
Path to clamd.pid:
clamd_pid="/var/run/clamav/clamd.pid"
Reload after update:
reload_dbs="yes" reload_opt="kill -USR2 `cat $clamd_pid`" #Signals PID to reload dbs
Work Directory:
work_dir="/var/lib/clamav"
And once you're done with the configuration, set the following to yes:
user_configuration_complete="yes"
Now we run the update script to check if the download works:
clamav-unofficial-sigs.sh
Now we a add the script to the root crontab to be run once a day:
crontab -e
Add the following line at the end of the root crontab:
00 04 * * * /usr/bin/clamav-unofficial-sigs.sh –c /etc/clamav-unofficial-sigs.conf &> /dev/null
11. Greylisting with Sqlgrey
Install sqlgrey:
apt-get install sqlgrey
touch clients_fqdn_whitelist.local && touch clients_ip_whitelist.local
Edit sqlgrey.conf to look like:
#########################
## SQLgrey config file ##
#########################
# Notes:
# - Unless specified otherwise commented settings are SQLgrey's defaults
# - SQLgrey uses a specific config file when called with -f <conf_file>
## Configuration files
conf_dir = /etc/sqlgrey
## Log level
# Uncomment to change the log level (default is normal: 2)
# nothing: O, errors only: 0, warnings: 1, normal: 2, verbose: 3, debug: 4
# loglevel = 2
## log categories can be fine-tuned,
# here are the log messages sorted by types and levels,
# (anything over the loglevel is discarded):
#
# grey : (0) internal errors,
# (2) initial connections, early reconnections,
# awl matches, successful reconnections, AWL additions,
# (3) smart decision process debug,
# whitelist: (2) whitelisted connections,
# (3) actual whitelist hit,
# (4) whitelists reloads,
# optin: (3) optin/optout global result
# (4) optin/optout SQL query results
# spam : (2) attempts never retried,
# mail : (1) error sending mails,
# (4) rate-limiter debug,
# dbaccess : (0) DB errors,
# (1) DB upgrade,
# (2) DB upgrade details,
# martians : (2) invalid e-mail addresses,
# perf : (2) cleanup time,
# system : (0) error forking,
# (3) forked children PIDs, children exits,
# conf : (0) errors in config files, missing required file,
# (1) warnings in config files,
# missing optional configuration files,
# (2) reloading configuration files,
# other : (4) Startup cleanup
# you can set a level to O (capital o) to disable logs completely,
# but be aware that then SQLgrey can come back to haunt you...
# Provide a coma-separated "logtype:loglevel" string
# For example if you set the loglevel to 3 (verbose) but want SQLgrey to be:
# . quiet for whitelists
# . normal for greylisting
# uncomment the following line.
# log_override = whitelist:1,grey:2
# By default, log_override is empty
## Log identification
# by default this is the process name. If you define the following variable
# SQLgrey will use whatever you set it to
#log_ident = sqlgrey
## username and groupname the daemon runs as
user = sqlgrey
group = sqlgrey
## Socket
# On which socket do SQLgrey wait for queries
# use the following if you need to bind on a public IP address
# inet =
<public_ip>:2501
# default :
# inet = 2501 # bind to localhost:2501
## PID
# where to store the process PID
# pidfile = /var/run/sqlgrey.pid
## Config directory
# where to look for other configuration files (whitelists)
confdir = /etc/sqlgrey
## Greylisting delays
# If you want to be really strict (RFC-wise) use these
# This is *not* recommended, you'll have false positives
# reconnect_delay = 15 # don't allow a reconnection before 15 minutes
# max_connect_age = 2 # don't allow a reconnection after 2 hours
# default: (based on real-life experience)
# reconnect_delay = 5
# max_connect_age = 24
connect_src_throttle = 5
## Auto whitelists settings
# default is tailored for small sites
awl_age = 60
group_domain_level = 2
# For bigger sites you may want
# a smaller awl_age and a bigger group_domain_level
# awl_age = 32 # AWL must be renewed at least once a month
# 32 > 31 (max delay between monthly newsletters)
# group_domain_level = 10 # wait for 10 validated adresses to add a whole
# domain in AWL
## Database settings
# instead of Pg below use "mysql" for MySQL, "SQLite" for SQLite
# any DBD driver is allowed, but only the previous 3 have been tested
db_type = mysql
db_name = sqlgrey
# Note: the following are not used with SQLite
db_host = localhost
db_port = default
db_user = sqlgrey
db_pass = password
db_cleandelay = 1800 # in seconds, how much time between database cleanups
clean_method = sync # sync : cleanup is done in the main process,
# delaying other operations
# async: cleanup is done in a forked process,
# it won't delay mail processing
# BEWARE: lockups have been reported
# and are still investigated
## X-Greylist header added?
# This adds delay, whitelist and autowhitelist information in the headers
prepend = 1
## Greylisting method:
# - full : greylist by IP address
# - classc : greylist by class C network. eg:
# 2.3.4.6 connection accepted if 2.3.4.145 did connect earlier
# - smart : greylist by class C network unless there is no reverse lookup
# or it looks like a home-user address
# Default is smart
# greymethod = smart
## Optin/Optout (see README.OPTINOUT for details)
# - none : everyone is greylisted (default)
# - optin : one must optin to have its (incoming) messages being greylisted
# - optout : one must optout to not have its messages being greylisted
optmethod = out
## SQLgrey return value.
# SQLgrey can tell Postfix to:
# - immediately reject a message with a temporary reject code
# - only do so if following rules would allow the message to pass
# The first choice will prevent Postfix from spending time evaluating
# potentially expensive rules.
# In some cases you may want following rules to be aware of the connection
# this.
#
# We can specify a different rejection strategy for the first connection
# attempt, and for early reconnections. 'immed' chooses immediate rejection
# 'delay' choose delayed rejection
#
# By default we use delay on first attempt
reject_first_attempt = immed
# Default for early reconnection is the value affected to reject_first_attempt
reject_early_reconnect = immed
## Update server
# where to get updates for whitelists
whitelists_host = sqlgrey.bouton.name
## Postmaster address
# who gets urgent notifications (DB is down for example)
# default or empty: don't send mail notifications
# admin_mail = [email protected]
Create the database:
mysql -u root -p
CREATE DATABASE /*!32312 IF NOT EXISTS*/ sglgrey;
GRANT ALL ON sqlgrey.* TO sqlgrey@localhost IDENTIFIED BY 'password';
FLUSH PRIVILEDGES;
Edit /etc/sqlgrey/sqlgrey.conf and update the database settings sections accordingly
Add the following to smtpd_recipient_restrictions:
check_policy_service inet:127.0.0.1:2501, permit
Create
vi /usr/sbin/sqlgrey
(You can download the file from http://downloads.howtoforge.com/spamsnake_ubuntu_9.10/sqlgrey.)
chmod +x /usr/sbin/sqlgrey
