How To Build A Spam Filtering Mail Gateway

Want to support HowtoForge? Become a subscriber!
 
Submitted by brian_goldberg (Contact Author) (Forums) on Thu, 2005-04-14 11:08. :: Anti-Spam/Virus

How To Build A Spam Filtering Mail Gateway

By Brian Goldberg - {brian AT carbonite D0T com}

www.carbonite.com

Spam has evolved from a nusance to a threat. SysAdms need a strategy for combating spam. You can use filtering software loaded on all your users computers, but that takes a lot of time to install and maitenance can be real drag on your time and resources. A better way is to use a centralised device which filters your spam before it enters your enterprise. This "Anti-Spam Gateway" is a lot easier to manage and maintain than individually installed client software. Additionally, it can be tuned to be a lot more effective.

Overview

1. Build bare-bones Linux server
a. Custom Configurations
b. Partitions
c. Firewall Option
d. Package Selection
e. LANG variable

2. Install Postfix Message Transfer Agent (MTA)
a. Disable sendmail
b. Install Postfix
c. Configure Postfix
d. Test Postfix
e. Configure for mail forwarding
f. Test again

3. Install Mailscanner
a. Install MailScanner Package
b. Initial MailScanner Configuration

4. Install Spamassassin
a. Install SpamAssassin
b. Configure SpamAssassin

5. Install ClamAV
a. Install ClamAV
b. Configure ClamAV
c. Test ClamAV

Step I - Build Bare-Bones Linux Server

I've used some of the fairly recent versions of RedHat Linux. Versions 8, 9 or Fedora should work fine. I choose the custom build using the GUI installer.

a. Custom User Configurations
Select the generic selections for keyboard, language and timezone.

b. Partitions
You should partition the server with at least this layout:
                /
/usr
/var
This will protect your server from runaway log files.

c. Firewall Configuration
I chose to select the "no firewall" option. I consider this device to be a traffic management device and not a security device. Upstream security should be handeld by an actual firewall. Of course, many may disagree with this and choose to load IPTables. Just make sure you have the right chains configured to allow traffic to flow properly.

d. Package Selection
When you get to the package selections, DE-SELECT EVERYTHING. Go back and choose only the following items:

Editors -> you'll need this to vi files
Development Tools -> you'll need this to compile software

Once the machine builds itself, it will reboot.

e. Fix LANG Variable
Once it reboots, we need to edit the LANG variable. RedHat's LANG variable setting of LANG="en_US.UTF-8" can cause compilation errors in some perl code used by MailScanner and SpamAssassin.

In Red Hat Linux you must edit the file /etc/sysconfig/i18n to change the lines:
        
LANG="en_US.UTF-8"
SUPPORTED="en_US.UTF-8:en_US:en"
To:
LANG="en_US"
SUPPORTED="en_US.UTF-8:en_US:en"
You then need to re-set and export the LANG variable:
                [root@titan sysconfig]# LANG='en_US'
[root@titan sysconfig]# export LANG

Step II - Install Postfix

I chose to use postifx instead of sendmail for my MTA. I like postfix because its configuration is very understandable. Also, I believe it is a bit more lightweight than sendmail.

a. Disable existing Sendmail services
Before you install postfix, you need to disable the existing sendmail items running on your Linux box.
                Service sendmail stop
chkconfig sendmail off
b. Install Postfix
Download postfix 2.1.5 from www.postfix.org and install as per this postfix document. Make sure you add the required records in passwd, group and aliases files. Postfix and Mailscanner will not work without them!

Accept all of the default settings when you "make install"

c. Configure Postfix
Postifx has two files which control most of its functionality. These are main.cf and master.cf.

Specific main.cf edits:
                myhostname = titan.corp.com
mydomain = corp.com
myorgin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain $mydomain
mynetwork_style = host
Note: some of these items need to be changed, while some only need to be uncommented.

d. Test Postfix Build
It is very importiant to test postfix now to make sure everything works.

Send an email to this mail server. You can telnet on port 25 to this box and manually send an email.

e. Configure Postfix to forward email
Since we do not want this device to be the final destination for our mail, we need to configure Postfix to forward all mail for our domain to our SMTP mail server. We need to make sure that only mail for our domain is forwarded, and mail for other domains is dropped (do not become a open mail relay - very bad!)

Edit this item in main.cf
                relay_domains = lab.net
This tells Postfix which domains it should relay mail. All mail destined for this doamin (and only this domain) will be forwarded to its remote SMTP server. You can put multiple domains here, just seperate them with a comma or whitespace.

Add line to end of main.cf
                transport_maps = hash:/etc/postfix/transport
This tells Postfix what method to use to resolve the destination address for relayed mail:

Add line to end of "/etc/postfix/transport"
                lab.net                smtp:[192.168.2.225]
This command specifically maps the domain "lab.net" to the IP address 192.168.2.225 and tells Postfix to use SMTP as the transport. All mail destined for lab.net which is relayed thru this Spam Gateway will be forwarded via SMTP to 192.168.2.225.

Then run command:
                postmap /etc/postfix/transport
This command builds the hash table/file which Posfix will use to forward mail. If you don't do this, it wont work.

Finally add this line to main.cf
                append_at_myorigin = no
These lines will make sure your Spam Gateway does not add any of its own header domain info to the mail as it passes thru.

f. Test Again
Stop and start postfix to make sure all changes take.
                postfix stop
postfix start
I know this is redundent, but you really should test your system again before installing MailScanner. Make sure that mail gets passed thru the system wihtout problem. If you do encounter a problem, it will be alot easier to fix it now than after you've installed MailScanner, SpamAssassin and ClamAV.

Step III - Install MailScanner


a. Install MailScanner
MailScanner installation is very easy to install. Just download the package from http://mailscanner.info. I use the version for RedHat/Mandrake.

Place the tar file in you directory of choice then run:
                tar zxvf MailScanner-.tar.gz
Run the install script:
                ./install.sh
Use chkconfig to make sure MailScanner is set for the proper run levels.
                chkconfig --list | grep MailScanner
You should see:
                MailScanner     0:off   1:off   2:on    3:on    4:on    5:on    6:off
Also, you'll need to disable postfix via chkconfig. MailScanner starts postfix itself.
                chkconfig postfix off
b. Configure MailScanner Settings

Updates to postfix's main.cf by adding this line:
        header_checks = regexp:/etc/postfix/header_checks 
In the file /etc/postfix/header_checks add this line:
        /^Received:/ HOLD
Here are the edits to Mailscanner - place / update in /etc/MailScanner/MailScanner.conf
        Run As User = postfix
Run As Group = postfix
Incoming Queue Dir = /var/spool/postfix/hold
Outgoing Queue Dir = /var/spool/postfix/incoming
MTA = postfix
Here's some file permissions changes you'll need to make:
            chown postfix.postfix /var/spool/MailScanner/incoming
chown postfix.postfix /var/spool/MailScanner/quarantine
Its a good idea to test the server now. Send a message to the remote server and see if it goes thru. It should, and then you can move to installing SpamAssassin.

Step IV - SpamAssassin

a. Install SpamAssassin
SpamAssassin is also very easy to install, however, you need to make sure you have the proper PERL modules installed. They are:
                        Digest::SHA1
HTML::Parser
Optional Modules:
                        MIME::Base64
DB_File
Net::DNS
Mail::SPF::Query
Time::HiRes
You can install SpamAssassin with:
                perl -MCPAN -e 'install Mail::SpamAssassin'
Then install
                Net::DNS
b. Configure SpamAssassin
You don't need to edit any of the SpamAssassin conf files because all of the configuration is done thru MailScanner.

In /etc/MailScanner/MailScanner.conf we will make these changes:
Change this line:
                Use SpamAssassin = no
to:
                Use SpamAssassin = yes
Update the SpamAssassin User State Dir setting:
                SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin
and then run commands:
                mkdir /var/spool/MailScanner/spamassassin
chown postfix.postfix /var/spool/MailScanner/spamassassin
Restart MailScanner to make changes stick.
                service MailScanner restart

Step V - ClamAV

a. Install ClamAV
Before you install ClamAV, you need to add the clamav user and group. You can do this as follows:
                groupadd clamav
useradd -g clamav -s /bin/false -c "Clam AntiVirus" clamav
Once this is done, you can build the software.
Open up the package:
                tar xvzf clamav-0.80.tar.gz
Generic build proceedure:
                ./configure
make
I encountered a problem with my RedHat Fedora Core 3 build which was fixed by using this command "ln -s /usr/lib/libidn.so.11.4.6 /usr/lib/libidn.so". See this web page for details: "http://kb.atmail.com/view_article.php?num=132&title=libidn.so:%20No%20such%20file%20or%20directory"
                make install
Now you need to load the perl modules for the ClamAV
                perl -MCPAN -e shell 
install Parse::RecDescent
install Inline
install Mail::ClamAV
b. Configure ClamAV and MailScanner Settings
In /usr/local/etc/clamd.conf make the following edits:

Add '#' in front of the word 'Example'

Do the same in /usr/local/etc/freshclam.conf

Now you need to update ClamAV's virus signature files
                        [root@titus]# freshclam
ClamAV update process started at Sat Jan 29 19:43:51 2005
main.cvd is up to date (version: 29, sigs: 29086, f-level: 3, builder: tomek)
daily.cvd is up to date (version: 691, sigs: 804, f-level: 4, builder: ccordes)
Update MailScanner's configuration file to use ClamAV
                                'Virus Scanners = clamav'
In MailScanner.conf, check the setting of 'Monitors for ClamAV Updates' to ensure it matches the location of your ClamAV virus database files. This should be "/usr/local/share/clamav/*.cvd".

Original location of this document: http://www.carbonite.com/guides/scannerbuild.html


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Onyxtacular (registered user) on Tue, 2012-12-11 18:02.

First of all thanks for the document.  If I wasn't a complete n00b with this it would be perfect.  I have Server 12.04 LTS, the package selection screen does not have editors or compiler tools listed.  and when I went under manual installation....well let's just said I am not ready for that yet. 

I request  more detail on how to choose the packages.

 

Thanks!

Submitted by Anonymous (not registered) on Mon, 2012-12-03 13:58.
Excellent !

But it is really more simple to download http://sourceforge.net/projects/osmailcleaner/ and just install it in 15 minutes ...

More easy to manage and update  in the future ...

Best regards 
Olivier

 

Submitted by TuxTux (not registered) on Wed, 2011-09-21 22:59.

 Would it work for CentOS 6?

Greetings!!!

Submitted by Anonymous (not registered) on Mon, 2009-10-26 13:00.

For some people it is intresting for whatever reason to use SpamAssin together with a Windows based smtp-proxy. You can achieve this with NoSpamProxy. They have a demo version for 30 days available under http://www.nospamproxy.de/nospamproxy.htm

Uwe

Submitted by Anonymous (not registered) on Tue, 2006-06-20 17:45.

Hi.

MailScanner uses ways to interface with Postfix which are completely unsupported.

In fact, they might change any time on Postfix updates and break MailScanner (or worse: cause data loss due to MailScanner's assumptions on how postfix accesses the data). Such breakages (so far, to my knowledge, without data loss possibility) have happened in the past and might happen again in the future.

So if you insist on using MailScanner instead of the much more robust amavisd-new (or other smtp-proxying scanners), you should at least be aware that you need to be very careful whenever you update either Postfix or MailScanner.

There has been a lot of discussion about this on the postfix mailinglists in the past.

Regards, Sven

Submitted by Anonymous (not registered) on Fri, 2006-04-28 00:14.
If you want a nice anti-spam setup with a how-to walkthrough, giving you a system integrated with a webmin-based administrative interface, that includes setup help for Bayesian filtering using spamassassin, please take a look at http://piratefish.blogspot.com
Submitted by Anonymous (not registered) on Thu, 2006-02-23 17:14.

Section II b. should be "mynetworks_style" not "mynetwork_style"

Submitted by Anonymous (not registered) on Sun, 2006-02-05 10:34.
It was said: "Its a good idea to test the server now. Send a message to the remote server and see if it goes thru. It should, and then you can move to installing SpamAssassin." It didn't go thru. What now?
Submitted by admin (registered user) on Sat, 2006-02-11 19:18.

Please go to the HowtoForge forums and post your problem there including any error messages you have got.

http://www.howtoforge.com/forums

You will find there many users that installed the howtos already and also many of the howto authors are answering support questions there too.

Submitted by Anonymous (not registered) on Thu, 2006-01-19 23:44.

I can understand your choice or ClamAV as virus scanner, because it is totally free and open source, but I have concerns about its ability to keep out the newest viruses (and that is the most important job for an AV program.

I found BitDefender offers free use (for all) on Linux systems as well as F-Prot and Antivir (for Linux home use only). They are all easy to install and are as good as each other and (sorry to say) better than clamAV in catching new viruses.

One could also use any commercial AV with unix support. I found Sophos works really well.

They all work well and are updated automatically (with little tweaking) by MailScanner.

Submitted by Anonymous (not registered) on Wed, 2012-08-01 17:57.
Careful, those antivirus programs are not free for commercial or for business environment use. Thus, you should not use them an a email gateway which is used in (or for) companies, in most cases. ClamAV is the only free and reliable antivirus - yet.
Submitted by Anonymous (not registered) on Wed, 2006-02-08 17:40.
I use ClamAV, Trend and Norton on my servers and I've found ClamAV is updated just as fast and in many cases faster then the commercial counterparts. Independant studies also support this, check out the ClamAV site for more info. Still you should never rely on this type of email gateway alone to prevent virus outbreaks. You need antivirus on all servers and workstations if you want to be safe.
Submitted by Anonymous (not registered) on Wed, 2005-12-21 07:12.

If you like this, but want something more robust and easier to setup, take a look at www.piratefish.org

It uses the same elements, but it's mostly web managed through webmin.

Submitted by Anonymous (not registered) on Tue, 2005-12-06 20:25.

Excelent document. I was looking for this kind of document the last week and finally I've found it. Nice help for a big problem.


Omar Rojas García

omarrojasg@yahoo.es

Submitted by Anonymous (not registered) on Wed, 2005-11-02 17:58.
When i did the mailscanner install, it did not place any scripts in the init.d directory, nor could i find any in the /opt/MailScanner dirs. the chkconfig command listed above yielded nothing whatsoever. I looked at the MailScanner page, but some of it's pages were missing on the site, and i couldn't find any mention of the init scripts required.