The Perfect SpamSnake - Ubuntu Jeos 9.10 - Page 5

Want to support HowtoForge? Become a subscriber!
 
Submitted by Rocky (Contact Author) (Forums) on Fri, 2010-04-16 12:42. ::

12. KAM

Create /etc/cron.daily/kam.sh with the following content:

#!/bin/bash
  
  # Original version modified by Andrew MacLachlan (andrew@gdcon.net)
  # Added additional MailScanner restarts on inital restart failure
  # Made script run silently for normal (successful) operation
  # Increased UPDATEMAXDELAY to 900 from 600
  
  # Insert a random delay up to this value, to spread virus updates round
  # the clock. 1800 seconds = 30 minutes.
  # Set this to 0 to disable it.
  UPDATEMAXDELAY=0
  if [ -f /opt/MailScanner/var/MailScanner ] ; then
  . /opt/MailScanner/var/MailScanner
  fi
  export UPDATEMAXDELAY
  
  if [ "x$UPDATEMAXDELAY" = "x0" ]; then
  :
  else
  logger -p mail.info -t KAM.cf.sh Delaying cron job up to $UPDATEMAXDELAY seconds
  perl -e "sleep int(rand($UPDATEMAXDELAY));"
  fi
  
  # JKF Fetch KAM.cf
  #echo Fetching KAM.cf...
  cd /etc/mail/spamassassin
  rm -f KAM.cf
  wget -O KAM.cf http://www.peregrinehw.com/downloads/SpamAssassin/contrib/KAM.cf > /dev/null 2>&1
  if [ "$?" = "0" ]; then
  #echo It completed and fetched something
  if ( tail -10 KAM.cf | grep -q '^#.*EOF' ); then
  # echo It succeeded so make a backup
  cp -f KAM.cf KAM.cf.backup
  else
  echo ERROR: Could not find EOF marker
  cp -f KAM.cf.backup KAM.cf
  fi
  else
  echo It failed to complete properly
  cp -f KAM.cf.backup KAM.cf
  fi
  #echo Reloading MailScanner and SpamAssassin configuration rules
  /etc/init.d/mailscanner reload > /dev/null 2>&1
  if [ $? != 0 ] ; then
  echo "MailScanner reload failed - Retrying..."
  /etc/init.d/mailscanner force-reload
  if [ $? = 0 ] ; then
  echo "MailScanner reload succeeded."
  else
  echo "Stopping MailScanner..."
  /etc/init.d/mailscanner stop
  echo "Waiting for a minute..."
  perl -e "sleep 60;"
  echo "Attemping to start MailScanner..."
  /etc/init.d/mailscanner start
  fi
  
  fi 

Make it executable

chmod +x /etc/cron.daily/kam.sh

 

13. ScamNailer

Create /opt/MailScanner/bin/update_scamnailer:

(You can download the script from http://downloads.howtoforge.com/spamsnake_ubuntu_9.10/update_scamnailer.)

Make it executable:

chmod +x /opt/MailScanner/bin/update_scamnailer

 

14. Cron jobs:

# m h  dom mon dow   command
30 0 * * * /opt/MailScanner/bin/update_phishing_sites #Update Phishing Sites
35 0 * * * /opt/MailScanner/bin/update_bad_phishing_sites #Update bad phishing sites
40 0 * * * /opt/MailScanner/bin/update_virus_scanners #Update virus scanners
@hourly /opt/MailScanner/bin/check_mailscanner #Make sure MailScanner is running
* 1 * * * /opt/MailScanner/bin/update_spamassassin #Update spamassassin
* 0 * * * /opt/MailScanner/bin/db_clean.php --clean #Clean Mailwatch DB
10 0 * * * /opt/MailScanner/bin/quarantine_maint.php --clean #Clean Quarantine
* * * * 6 /usr/sbin/update_sqlgrey_config #Update Sqlgrey Config
50 0 * * * /opt/MailScanner/bin/update_scamnailer #Update Scamnailer
* 23 * * * /usr/bin/clamav-unofficial-sigs.sh #Update Sanesecurity

 

15. Firewalling the SpamSnake with Firehol

Introduction

Firehol is a stateful iptables packet filtering firewall configurator. It is abstracted, extensible, easy and powerful. It can handle any kind of firewall, but most importantly, it gives you the means to configure it, the same way you think of it.

 

Install Firehol

apt-get install firehol

 

Firehol Settings

Edit /etc/default/firehol and change the following:

START_FIREHOL=YES

Edit /etc/firehol/firehol.conf and add the following:

version 5
# Accept all client traffic on any interface
interface any internet
protection strong
server "icmp ping ICMP ssh http https telnet webmin dns dcc echo smtp" accept
client all accept

Be sure to comment out the default configuration before applying these settings. This filters all incoming connections that are not related to the above services.
If you want to be less polite, you can drop them by adding the following after 'protection strong': policy drop

 

Updating RESERVED_IPS list

cd /usr/src && wget http://firehol.sf.net/firehol.tar.gz
tar xzvf firehol.tar.gz && cd firehol
mv get-iana.sh /usr/bin/get-iana.sh
chmod +x /usr/bin/get-iana.sh

Run the script to update RESERVED_IPS:

get-iana.sh

Make sure to select 'yes' when asked if you would like to save RESERVED_IPS to /etc/firehol/RESERVED_IPS.

Start Firehol:

/etc/init.d/firehol start

Now your server is set up to only accept connections for the services you allowed.

 

16. Apply Relay Recipients

The following directions are meant for people using Microsoft Exchange 2000 or Microsoft Exchange 2003.

This page describes how to configure your mail gateway to periodically get a list of valid recipient email addresses from your Exchange system. By doing this, you can configure your server to automatically reject any email addressed to invalid addresses. This will reduce the load on your exchange server, since it no longer has to process non-delivery reports, and it will reduce the load on your postfix server since it won't have to perform spam and virus scanning on the message.

 

Install Dependencies

Install the perl module Net::LDAP:

perl -MCPAN -e shell
install Net::LDAP

Create the Get Email Address Script:

vi /usr/bin/getadsmtp.pl

Copy and paste the code below into this new file.

#!/usr/bin/perl -T -w
# This script will pull all users' SMTP addresses from your Active Directory
# (including primary and secondary email addresses) and list them in the
# format "user@example.com OK" which Postfix uses with relay_recipient_maps.
# Be sure to double-check the path to perl above.
# This requires Net::LDAP to be installed.  To install Net::LDAP, at a shell
# type "perl -MCPAN -e shell" and then "install Net::LDAP"
use Net::LDAP;
use Net::LDAP::Control::Paged;
use Net::LDAP::Constant ( "LDAP_CONTROL_PAGED" );
# Enter the path/file for the output
$VALID = "/etc/postfix/relay_recipients";
open VALID, ">$VALID" or die "CANNOT OPEN $VALID $!";
# Enter the FQDN of your Active Directory domain controllers below
$dc1="domaincontroller1.example.com";
$dc2="domaincontroller2.example.com";
# Enter the LDAP container for your userbase.
# The syntax is CN=Users,dc=example,dc=com
# This can be found by installing the Windows 2000 Support Tools
# then running ADSI Edit.
# In ADSI Edit, expand the "Domain NC [domaincontroller1.example.com]" &
# you will see, for example, DC=example,DC=com (this is your base).
# The Users Container will be specified in the right pane as
# CN=Users depending on your schema (this is your container).
# You can double-check this by clicking "Properties" of your user
# folder in ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=Users,DC=example,DC=com
# which would be $hqbase="cn=Users,dc=example,dc=com"
# Note:  You can also use just $hqbase="dc=example,dc=com"
$hqbase="cn=Users,dc=example,dc=com";
# Enter the username & password for a valid user in your Active Directory
# with username in the form cn=username,cn=Users,dc=example,dc=com
# Make sure the user's password does not expire.  Note that this user
# does not require any special privileges.
# You can double-check this by clicking "Properties" of your user in
# ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=user,CN=Users,DC=example,DC=com
# which would be $user="cn=user,cn=Users,dc=example,dc=com"
# Note: You can also use the UPN login: "user\@example.com"
$user="cn=user,cn=Users,dc=example,dc=com";
$passwd="password";
# Connecting to Active Directory domain controllers
$noldapserver=0;
$ldap = Net::LDAP->new($dc1) or
   $noldapserver=1;
if ($noldapserver == 1)  {
   $ldap = Net::LDAP->new($dc2) or
      die "Error connecting to specified domain controllers $@ \n";
}
$mesg = $ldap->bind ( dn => $user,
                      password =>$passwd);
if ( $mesg->code()) {
    die ("error:", $mesg->error_text((),"\n"));
}
# How many LDAP query results to grab for each paged round
# Set to under 1000 for Active Directory
$page = Net::LDAP::Control::Paged->new( size => 990 );
@args = ( base     => $hqbase,
# Play around with this to grab objects such as Contacts, Public Folders, etc.
# A minimal filter for just users with email would be:
# filter => "(&(sAMAccountName=*)(mail=*))"
         filter => "(& (mailnickname=*) (| (&(objectCategory=person)
                    (objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))
                    (&(objectCategory=person)(objectClass=user)(|(homeMDB=*)
                    (msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=contact))
                    (objectCategory=group)(objectCategory=publicFolder) ))",
          control  => [ $page ],
          attrs  => "proxyAddresses",
);
my $cookie;
while(1) {
  # Perform search
  my $mesg = $ldap->search( @args );
# Filtering results for proxyAddresses attributes
  foreach my $entry ( $mesg->entries ) {
    my $name = $entry->get_value( "cn" );
    # LDAP Attributes are multi-valued, so we have to print each one.
    foreach my $mail ( $entry->get_value( "proxyAddresses" ) ) {
     # Test if the Line starts with one of the following lines:
     # proxyAddresses: [smtp|SMTP]:
     # and also discard this starting string, so that $mail is only the
     # address without any other characters...
     if ( $mail =~ s/^(smtp|SMTP)://gs ) {
       print VALID $mail." OK\n";
     }
    }
  }
  # Only continue on LDAP_SUCCESS
  $mesg->code and last;
  # Get cookie from paged control
  my($resp)  = $mesg->control( LDAP_CONTROL_PAGED ) or last;
  $cookie    = $resp->cookie or last;
  # Set cookie in paged control
  $page->cookie($cookie);
}
if ($cookie) {
  # We had an abnormal exit, so let the server know we do not want any more
  $page->cookie($cookie);
  $page->size(0);
  $ldap->search( @args );
  # Also would be a good idea to die unhappily and inform OP at this point
     die("LDAP query unsuccessful");
}
# Add additional restrictions, users, etc. to the output file below.
#print VALID "user\@domain1.com OK\n";
#print VALID "user\@domain2.com 550 User unknown.\n";
#print VALID "domain3.com 550 User does not exist.\n";
close VALID;

Next set the permissions on the file to allow it to be executed:

chmod 500 /usr/bin/getadsmtp.pl

Edit the file to customize it for your specific domain. Since the file is read only, you will need to use :w! to save the file in vi.

1. Set $dc1 and $dc2 to the fully qualified domain names or IP addresses of 2 of your domain controllers.
2. Set $hqbase equal to the LDAP path to the container or organizational unit which holds the email accounts for which you wish to get the email addresses.
3. Set $user and $passwd to indicate which user account should be used to access this information. This account only needs to be a member of the domain, so it would be a good idea to setup an account specifically for this.

Try running the script. If it works correctly, it will create /etc/postfix/relay_recipients.

Note that if your postfix server is separated from your active directory controllers by a firewall, you will need to open TCP port 389 from the postfix server to the ADCs.

At this point, you can update your /etc/postfix/main.cf to relay_recipient_maps. You will also have to postmap the file to create the database.

getadsmtp.pl

At this point, you may want to edit /etc/postfix/relay_recipients and edit out any unwanted email addresses as this script imports everything.

 

Create the Table

postmap /etc/postfix/relay_recipients

Finally, you may want to set up a cron job to periodically update and build the /etc/postfix/relay_recipients.db file. You can set up a script called /usr/bin/update-relay-recipients.sh: (Optional)

vi /usr/bin/update-relay-recipients.sh

#!/bin/sh
/usr/bin/getadsmtp.pl
cd /etc/postfix
postmap relay_recipients

Don't forget to make sure the following is in your /etc/postfix/main.cf file:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

Make the script excutable:

chmod +x /usr/bin/update-relay-recipients.sh

Run crontab to add this script to the scheduled jobs:

crontab -e

Now add the following lines to the bottom of the file. Note that this cron job will run every day at 2:30 AM to update the database file. You may want to run yours more frequently or not depending on how often you add new email users to your system.

# syncronize relay_recipients with Active Directory addresses
30 2 * * * /usr/bin/update-relay-recipients.sh

 

17. Install Webmin (Optional):

apt-get install perl libnet-ssleay-perl libauthen-pam-perl libio-pty-perl libmd5-perl
cd /usr/src && wget http://downloads.sourceforge.net/project/webadmin/webmin/1.500/webmin_1.500_all.deb?use_mirror=iweb dpkg --install webmin_1.500_all.deb

 

18. Automatically Add A Disclaimer To Outgoing Emails With alterMIME (Optional)

This tutorial shows how to install and use alterMIME. alterMIME is a tool that can automatically add a disclaimer to emails. In this article I will explain how to install it as a Postfix filter on Ubuntu.

Installing alterMIME:

apt-get install altermime

Next we create the user filter with the home directory /var/spool/filter - alterMIME will be run as that user:

useradd -r -c "Postfix Filters" -d /var/spool/filter filter
mkdir /var/spool/filter
chown filter:filter /var/spool/filter
chmod 750 /var/spool/filter

Afterwards we create the script /etc/postfix/disclaimer which executes alterMIME. Ubuntu's alterMIME package comes with a sample script that we can simply copy to /etc/postfix/disclaimer:

cp /usr/share/doc/altermime/examples/postfix_filter.sh /etc/postfix/disclaimer
chgrp filter /etc/postfix/disclaimer
chmod 750 /etc/postfix/disclaimer

Now the problem with this script is that it doesn't distinguish between incoming and outgoing emails - it simply adds a disclaimer to all mails. Typically you want disclaimers only for outgoing emails, and even then not for all sender addresses. Therefore I've modified the /etc/postfix/disclaimer script a little bit - we'll come to that in a minute.

Right now, we create the file /etc/postfix/disclaimer_addresses which holds all sender email addresses (one per line) for which alterMIME should add a disclaimer:

vi /etc/postfix/disclaimer_addresses

user1@example.com
user2@example.org
user3@example.net

Now we open /etc/postfix/disclaimer and modify it as follows (I have marked the parts that I've changed):

vi /etc/postfix/disclaimer

#!/bin/sh
# Localize these.
INSPECT_DIR=/var/spool/filter
SENDMAIL=/usr/sbin/sendmail
####### Changed From Original Script #######
DISCLAIMER_ADDRESSES=/etc/postfix/disclaimer_addresses
####### Changed From Original Script END #######
# Exit codes from <sysexits.h>
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
# Clean up when done or when aborting.
trap "rm -f in.$$" 0 1 2 3 15
# Start processing.
cd $INSPECT_DIR || { echo $INSPECT_DIR does not exist; exit
$EX_TEMPFAIL; }
cat >in.$$ || { echo Cannot save mail to file; exit $EX_TEMPFAIL; }
####### Changed From Original Script #######
# obtain From address
from_address=`grep -m 1 "From:" in.$$ | cut -d "<" -f 2 | cut -d ">" -f 1`
if [ `grep -wi ^${from_address}$ ${DISCLAIMER_ADDRESSES}` ]; then
  /usr/bin/altermime --input=in.$$ \
                   --disclaimer=/etc/postfix/disclaimer.txt \
                   --disclaimer-html=/etc/postfix/disclaimer.txt \
                   --xheader="X-Copyrighted-Material: Please visit http://www.company.com/privacy.htm" || \
                    { echo Message content rejected; exit $EX_UNAVAILABLE; }
fi
####### Changed From Original Script END #######
$SENDMAIL "$@" <in.$$
exit $?

Next we need the text file /etc/postfix/disclaimer.txt which holds our disclaimer text. Ubuntu's alterMIME package comes with a sample text that we can use for now (of course, you can modify it if you like):

cp /usr/share/doc/altermime/examples/disclaimer.txt /etc/postfix/disclaimer.txt

Finally we have to tell Postfix that it should use the /etc/postfix/disclaimer script to add disclaimers to outgoing emails. Open /etc/postfix/master.cf and add -o content_filter=dfilt: to the smtp line:

vi /etc/postfix/master.cf

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
   -o content_filter=dfilt:
[...]

At the end of the same file, add the following two lines:

[...]
dfilt     unix    -       n       n       -       -       pipe
    flags=Rq user=filter argv=/etc/postfix/disclaimer -f ${sender} -- ${recipient} 

Restart Postfix afterwards:

/etc/init.d/postfix restart

That's it! Now a disclaimer should be added to outgoing emails sent from the addresses listed in /etc/postfix/disclaimer_addresses.

 

Congratulations

You should now have a complete working SpamSnake.


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by iser0073 (not registered) on Mon, 2010-08-16 22:00.

 the get-iana.sh script is giving me the following problems:

====================================================
aggregate: no prefixes supplied


FOUND THE FOLLOWING RESERVED IP RANGES:
RESERVED_IPS=""


Failed to find reserved IPs.
Possibly the file format has been changed, or I cannot fetch the URL.
====================================================

To fix this problem I had to edit /usr/bin/get-iana.sh file and make the following changes:

  1.  Replace the line :
    IPV4_ADDRESS_SPACE_URL="http://www.iana.org/assignments/ipv4-address-space"
    with this one :
    IPV4_ADDRESS_SPACE_URL="http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.txt"
  2. Replace the line :
    cut -d ' ' -f 1 |\
    with this one :
    awk '{print $1}' |\

now run the script again :
get-iana.sh

This fixed the problem for me.

I have also uploaded my working script, maybe someone can use my version and not have to go through all the trouble, you can grab it from here:
http://ftp.bitfarm.co.za/Pics/iser0073/get-iana.sh

Remember to make the file executable after downloading :
chmod +x /usr/bin/get-iana.sh

Hope this helps a little.

Thanx for a Brilliant tutorial!!

Submitted by 011 (not registered) on Fri, 2010-04-23 03:13.

I am not sure why only 2000 and 2003 exchange would be of interest here. 2000 version will take any email and then fiind out i fthere is a valid address so script might be of use. 2003 and later exchange will declined delivery of receipient is not valid.

Does this script retreive aliases too?

I am sure that postfix has capability to check if receipient is valid by asking exchenge 2003 and 2007 is email address is valid. I have a note that it is by adding, in main.cf, some check_receipient_address and adress_verify_map configurations

Server like 2000 exchange are rear. Why bother with script?

Submitted by Peter (not registered) on Mon, 2010-04-26 14:13.
The point of the Relay Recipients is to drop the connection at the SpamSnake if it is sent to an invalid recipient.  Why bother tying up resources on your mail server checking to see if the recipient is valid?  This is actually a great feature.  I would estimate 30-40% of the messages we receive are sent to invalid recipients.  Using a previous version of this SpamSnake has reduced the load on our mail server dramatically.