HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials

HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials (http://www.howtoforge.com/forums/index.php)

- Tips/Tricks/Mods (http://www.howtoforge.com/forums/forumdisplay.php?f=29)

- - HOWTO: Spam control for POSTFIX (http://www.howtoforge.com/forums/showthread.php?t=48608)

HOWTO: Spam control for POSTFIX

Spam is a major problem for anyone with a mail server. Many times, spam goes to email addresses that don't exist. But, it still is hitting your server even if it isn't delivered. Other times, a users inbox will be overflowing with annoying messages about Viagra, hookers, free software, and whatever else.

Below is a solution. It's an ongoing accumulation of my efforts to stop spam to the best of my abilities. So far, it has a 97% success rate with over 20,000 emails (spam and ham, alike) processed.

Follow the instructions. I will update/modify as necessary if things seem unclear. Feel free to ask.

1) Install Postgrey, RRD, a log parser, and Graphing tools.

Code:

apt-get install postgrey rrdtool mailgraph pflogsumm

Postgrey will have a delay of 5 minutes by default on email going to your mailbox. If this is too long, edit the /etc/default/postgrey file by adding "--delay=120" where 120 is seconds.

2) Restart the Postgrey server.

Code:

/etc/init.d/postgrey restart

3) edit the Postfix main.cf. We will be adding several things including the Postgrey configuration.

Code:

# See /usr/share/postfix/main.cf.dist for a commented, more complete version





# Debian specific:  Specifying a file name will cause the first

# line of that file to be used as the name.  The Debian default

# is /etc/mailname.

#myorigin = /etc/mailname



smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)

biff = no



# appending .domain is the MUA's job.

append_dot_mydomain = no



# Uncomment the next line to generate "delayed mail" warnings

#delay_warning_time = 4h



readme_directory = /usr/share/doc/postfix



# TLS parameters

smtpd_tls_cert_file = /etc/postfix/smtpd.cert

smtpd_tls_key_file = /etc/postfix/smtpd.key

smtpd_use_tls = yes

smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache

smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache



# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for

# information on enabling SSL in the smtp client.



myhostname = my.derekgordon.com

alias_maps = hash:/etc/aliases

alias_database = hash:/etc/aliases

myorigin = /etc/mailname

mydestination = my.derekgordon.com, localhost, localhost.localdomain

relayhost = 

mynetworks = 127.0.0.0/8 [::1]/128 66.118.142.78

mailbox_command = procmail -a "$EXTENSION"

mailbox_size_limit = 0

recipient_delimiter = +

inet_interfaces = all

html_directory = /usr/share/doc/postfix/html

virtual_alias_domains = 

virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, mysql:/etc/postfix/mysql-virtual_email2email.cf

virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf

virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf

virtual_mailbox_base = /var/vmail

virtual_uid_maps = static:5000

virtual_gid_maps = static:5000

smtpd_sasl_auth_enable = yes

broken_sasl_auth_clients = yes

smtpd_sasl_authenticated_header = yes

smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, check_recipient_access mysql:/etc/postfix/mysql-virtual_recipient.cf, reject_unauth_destination, check_policy_service inet:127.0.0.1:60000, reject_rbl_client zen.spamhaus.org, reject_rbl_client smtp.dnsbl.sorbs.net, reject_rbl_client bl.spamcop.net, reject_rbl_client combined.rbl.msrbl.net, check_recipient_access regexp:/etc/postfix/spamtrap, permit

smtpd_tls_security_level = may

transport_maps = proxy:mysql:/etc/postfix/mysql-virtual_transports.cf

relay_domains = mysql:/etc/postfix/mysql-virtual_relaydomains.cf

relay_recipient_maps = mysql:/etc/postfix/mysql-virtual_relayrecipientmaps.cf

virtual_create_maildirsize = yes

virtual_maildir_extended = yes

virtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailbox_limit_maps.cf

virtual_mailbox_limit_override = yes

virtual_maildir_limit_message = "The user you are trying to reach is over quota."

virtual_overquota_bounce = yes

proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps

smtpd_sender_restrictions = check_sender_access mysql:/etc/postfix/mysql-virtual_sender.cf

smtpd_client_restrictions = check_client_access mysql:/etc/postfix/mysql-virtual_client.cf

maildrop_destination_concurrency_limit = 1

maildrop_destination_recipient_limit = 1

virtual_transport = maildrop

header_checks = regexp:/etc/postfix/header_checks

mime_header_checks = regexp:/etc/postfix/mime_header_checks

nested_header_checks = regexp:/etc/postfix/nested_header_checks

body_checks = regexp:/etc/postfix/body_checks

content_filter = amavis:[127.0.0.1]:10024

receive_override_options = no_address_mappings

message_size_limit = 0



disable_vrfy_command = yes

smtpd_helo_required = yes

smtpd_helo_restrictions = permit_mynetworks, reject_invalid_hostname, permit

From a generic main.cf found in the Debian Lenny installation, I added/modified the BOLD areas.

4) Create a file named "spamtrap" in the /etc/postfix/ directory. This serves as a filter. If spam is emailed to this address and other addresses on your machine, it will drop that email so that it doesn't get to any other mailboxes.

The Spamtrap file looks like this:

Code:

/emailcontrol.*@derekgordon\.com/ DISCARD

This is regexp so the slashes have to be used. My filter email is emailcontrol@derekgordon.com so edit accordingly and place in the spamtrap file!!!

Side note: Do not create this mailbox using ISPConfig. There is absolutely no reason for it to exist on your mailserver. It's a fake address meant to catch annoying spam.

5) Open up local.cf for SpamAssassin and add the following bit. It will be an extra filter designed to work with SA more than it is with general Postfix.

Code:

nano /etc/spamassassin/local.cf

Add the following to the bottom:

Code:

urirhssub       URIBL_BLACK  multi.uribl.com.        A   2

body            URIBL_BLACK  eval:check_uridnsbl('URIBL_BLACK')

describe        URIBL_BLACK  Contains an URL listed in the URIBL blacklist

tflags          URIBL_BLACK  net       

score           URIBL_BLACK  3.0

6) Restart Postfix and Spamassassin

Code:

/etc/init.d/postfix restart

Code:

/etc/init.d/spamassassin restart

7) Copy the mailgraph CGI script to your websites CGI-BIN:

Code:

cp -p /usr/lib/cgi-bin/mailgraph.cgi /var/www/www.example.com/cgi-bin

8) Create and CHMOD the postfix_report.sh script:

Code:

nano /usr/local/sbin/postfix_report.sh

Paste the following into the script:

Code:

#!/bin/sh

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin



pflogsumm /var/log/mail.log | formail -c -I"Subject: Mail Statistics" -I"From: maillog@example.com" -I"To: youremail@yourdomain.com" -I"Received: from www.example.com ([ 127.0.0.1])" | sendmail youremail@yourdomain.com



##gzip /var/log/mail.log.0

exit 0

Code:

chmod 755 /usr/local/sbin/postfix_report.sh

9) Edit the RSYSLOG file so that your mail.log rotates daily and to setup an automatic email with postfix statistics:

Code:

nano /etc/logrotate.d/rsyslog

Delete the line that says "/var/log/mail.log" and add this at the VERY bottom of the file:

Code:

/var/log/mail.log

{

        rotate 7

        daily

        missingok

        notifempty

        delaycompress

        compress

        prerotate

              /usr/local/sbin/postfix_report.sh > /dev/null

        endscript

        postrotate

                invoke-rc.d rsyslog reload > /dev/null

        endscript

}

With this, every time the mail.log rotates (usually around 6am by default) you will get a detailed email about what Postfix has delivered, not delivered, greylisted, and so on.

So now you're all done! What did you do? You installed blacklist filters, greylisting, graphing for on-the-fly information about Postfix, daily emails with detailed Postfix stats, created a spam trap, and other minor things to make your mailserver a lot more secure and less susceptible to spam.

IMPORTANT: Let me know what you all do. Please respond with your choice, if you use it, and how well it worked. If there's much of a use, I will keep building upon the instructions and make it even better (hopefully). Responses are in the form of thread messages.

----------------------

BONUS INSTRUCTIONS: If you use the script I posted below, that gives you GREYLISTING SPECIFIC STATS, do the following:

1) DELETE it from having a CRONJOB if you added one originaly. Mostlikely, you did.

2) Open the RSYSLOG file again.

3) Modify the above entry so that it looks like this:

Code:

/var/log/mail.log

{

        rotate 7

        daily

        missingok

        notifempty

        delaycompress

        compress

        prerotate

              /path/to/the/greylist_script.sh > /dev/null

              /usr/local/sbin/postfix_report.sh > /dev/null

        endscript

        postrotate

                invoke-rc.d rsyslog reload > /dev/null

        endscript

}

Make sure that the "/path/to/the/greylist_script.sh > /dev/null" matches the exact path to the script you were using.

Again, That script is found a few posts below.

---------------------------------------
TIPS AND FIXES:

1) If you have people sending you emails and they get rejected because SORBS says their dynamic IP is a bad one, change the following in /etc/postfix/main.cf: "reject_rbl_client dnsbl.sorbs.net" becomes "reject_rbl_client smtp.dnsbl.sorbs.net"
SORBS has a huge variation in repositories. Some AT&T DSL IP's will be found in the DNSBL it appears. So, that could cause a potential no deliver.

2) Similarly to #1, if you find out that there are some SORBS errors in delivering legit emails, you may want to experiement with the various Zones. https://www.au.sorbs.net/using.shtml lists all of them. I recommend SMTP, NOMAIL, SPAM, and ZOMBIE.

3) When upgrading ISPC3 to later versions, the main.cf for Postfix will be erased. This means you can either backup the one you created and replace the changes made by the update script for ISP3, or you can just copy the main.cf information above and use it again. This is the only step that has to be repeated. So, again if you upgrade ISPC3, you need to reconfigure the main.cf for Postfix as it will be replaced by the ISPC3 default main.cf again.

4) If you used this guide prior to 13 October 2010, I recommend removing "reject_rbl_client multihop.dsbl.org" from the main.cf for Postfix. That database has been taken offline. This was "effective" in 2009, but still worked until recently.

The next big step for fixing spam issues is to build a robust header_check and body_check file for Postfix.

As you see above, I have started on the header_check.

I'd like to determine how to setup a SUBJECT filter that would delete any email with more than 7 words. I have had NO legitimate email with more than 6 words in the subject. However, a lot of spam has between 7 and 12 words in the SUBJECT line.

I'm not finding great details on how it works, so it's going to take a lot of time. If you have any of your own mod's please post them so I can compile a better post above.

Also, filtering out all languages you DO NOT use would be sufficient. I will be filtering out all languages except ENGLISH, SPANISH, FRENCH, GERMAN, CZECH, POLISH, and PORTUGUESE when I have time.

Quote:

The next big step for fixing spam issues is to build a robust header_check and body_check file for Postfix.

Thats implemented in ispconfig, so do not add such a file manually to ensure that you will not get conflicts. Use the header and body filter function from ispconfig instead.

By the way, I wont use such filter files for spam filtering. The normal spamassassin and blacklists filter already about 99.x % of all spam and adding such manual filters will just rise the risk of false positives. You can not win the battle against sapm with manual filter rules.

Yes, but if you have NO header blocks added in ISPCONFIG, you can use the header file directly.

In cases that it's not a production server, just for personal uses, it might be easier to copy/paste the above header_check.

However, I understand your point and if entries were being made it would be a conflict.

So, I would recommend doing what Till says if you're already using that feature or may use that feature.

I removed the HEADER_CHECKS I created from the HOWTO as I don't want to confuse.

Here they are for you expert experimenters.

Last piece, and this will be updated as I get more in depth, header_check in the postfix directory should contain these:

Code:

/^Subject: .*       / REJECT Spam Header Many Spaces 1



# Chinese, Japanese and Korean



/^Content-Type:.*?charset\s*=\s*�?(Big5|gb2312|euc-cn)�?/ REJECT HDR2100: Unaccepted character set: �$1?

/^Content-Type:.*?charset\s*=\s*�?(euc-kr|iso-2022-kr)�?/ REJECT HDR2110: Unaccepted character set: �$1?

/^Content-Type:.*?charset\s*=\s*�?(iso-2022-\w+|euc-jp|shift_jis)�?/ REJECT HDR2120: Unaccepted character set: �$1?





# Cyrrilic character sets: Russian/Ukrainian



/^Content-Type:.*?charset\s*=\s*�?(koi8-(?:r|u))�?/ REJECT HDR2200: Unaccepted character set: �$1?

/^Content-Type:.*?charset\s*=\s*�?(windows-(?:1250|1251))�?/ REJECT HDR2210: Unaccepted character set: �$1?

I will continue to build the header_check file as time goes on. So stay tuned.

Question/Recap:
- Till is saying not to perform header/body checks within main.cf and use the tools within ispc3, yes? If so, that makes perfect sense to me. I wish there was a service/blocklist that already did it on a realtime basis (somehow).

- Is Postgrey good to load or no? I still need to research postgrey to figure out what it is.

Sorry for my confusion and thank you for including me. By you posting your main.cf, I noticed I had limited my sorbs bl. By changing to yours I blocked a lot more immedietly.

Yes, I would do header_checks in ISPC3 if you're comfortable with that. It's pretty simple in the end. I haven't tested my new ones I wrote yet, so won't post them. But, I'm thinking that postgrey will be the best solution.

POSTGREY is an easy implementation of Greylisting:

Code:

Greylisting is a new method of blocking significant amounts of spam at the mailserver level, but without resorting to heavyweight statistical analysis or other heuristical (and error-prone) approaches. Consequently, implementations are fairly lightweight, and may even decrease network traffic and processor load on your mailserver.



Greylisting relies on the fact that most spam sources do not behave in the same way as "normal" mail systems. Although it is currently very effective by itself, it will perform best when it is used in conjunction with other forms of spam prevention. For a detailed description of the method, see the Whitepaper.



The term Greylisting is meant to describe a general method of blocking spam based on the behavior of the sending server, rather than the content of the messages. Greylisting does not refer to any particular implementation of these methods. Consequently, there is no single Greylisting product. Instead, there are many products that incorporate some or all of the methods described here.

Postgrey specific:

Code:

Postgrey is a Postfix policy server implementing greylisting.  Development of Postgrey started at the ISG.EE and is now sponsored by Open Systems AG.



When a request for delivery of a mail is received by Postfix via SMTP, the triplet CLIENT_IP / SENDER / RECIPIENT is built. If it is the first time that this triplet is seen, or if the triplet was first seen, less than 5 minutes ago, then the mail gets rejected with a temporary error. Hopefully spammers or viruses will not try again later, as it is however required per RFC.

I have POSTGREY running. It's blocked a significant amount of unwanted email. It's only downside, if there is any, that it delays mail delivery by 5 minutes default.

Once you're using Postgrey and what to see how it's working, install the following script somewhere on your server:

"postgrey_stats.sh"

Code:

#!/bin/sh



LOGFILE=/tmp/greylist-statistics

YOURMAIL=youremail@yourdomain.com

 

echo "Total amount of GreyListed messages

" > $LOGFILE

cat /var/log/mail.log | /usr/bin/postgreyreport --delay=300 >> $LOGFILE

echo -ne  "-------------------------------------\n" >> $LOGFILE

echo -ne  "-------------------------------------\n" >> $LOGFILE

echo "Get only the top 20 sources getting greylisted out

" >> $LOGFILE

cat /var/log/mail.log | postgreyreport | awk '{print $1}' | sort | uniq -c | sort -nr | head -n20 >> $LOGFILE

echo -ne  "-------------------------------------\n" >> $LOGFILE

echo -ne  "-------------------------------------\n" >> $LOGFILE

echo "Get a list of the top 20 email address that the greylisted sources are sending email to

"  >> $LOGFILE

cat /var/log/mail.log | postgreyreport | awk '{print $4}'  | sort  | uniq -c | sort -nr | head -n20 >> $LOGFILE

echo -ne  "-------------------------------------\n" >> $LOGFILE

echo -ne  "-------------------------------------\n" >> $LOGFILE

cat $LOGFILE | mail -s "Greylisting Statistics of `hostname` for `date +%Y-%m-%d`" $YOURMAIL

Edit the following parts of the above script:

1) Change the "YOUREMAIL = " line so that it goes to your personal mail box. This will give you details on how its working.
2) Make sure that /var/log/mail.log is the correct path to your current mail.log file. Distros are different.

Set the script to chmod +700 so that it is executable.

Finally, create a cronjob so that it will give you an automatic output to your email daily (or whenever you choose).

Have you seen sqlgrey? Not sure if it's better or not but it has an interface that's interesting. Any thoughts?

I have seen it. However, I will stick with Postgrey for the time being.

I recommended in a REQUEST that Till research including SqlGrey and its interface in a future release. At that point, it wouldn't be necessary to use Postgrey.

But, Postgrey is extremely simple (see above instructions) so it isn't worth the extra hassle in my opinion for roughly the same functionality.

I'm researching an addon to Postgrey that will make it even better.

Give Postgrey a chance, use that script above to see how well it works (you will be shocked after the first full day), and see what happens.

The only false positive I have heard of was Paypal. But, that's unconfirmed and worked without a hitch for me.

You can add whitelists for domains, ips, specific emails, clients, etc... with easy in the event there were problems. Whitelists also will skip the delay in delivery.

Let me know what you all do. Please respond with your choice, if you use it, and how well it worked. If there's much of a use, I will keep building upon the instructions and make it even better (hopefully).