How To Automate Spamcop Submissions
How To Automate Spamcop Submissions
Author: Stephan Jau
Spamcop is a service which provides RBLs for mailservers in order to reject incoming mail from spammers.
Their philosophy is to process possible spam complaints from users. When they receive a certain amount of complaints during a time-period then they will blacklist the offender. This system is dependant on spam reporting from users. However, their submission process is not very user-friendly:
1.) You need either to forward the spam to a spamcop-email address given to you during sign-up (something like submit.aASdfafaASDf@spam.spamcop.net --> I made this address just up) or you manually copy'n'paste the headers and the body and fill them into a form on their server.
2.) You then receive an email to the email address that you have supplied when you signed up.
3.) In this email, there will be a link that asks you to verify the supplied data on a webform. This is not just clicking the link but also manually submitting the webpage.
As I have said above, Spamcop is pretty much dependant on the user input. If no one submits and verifies spam, then they will have no blacklist. However that whole submission and verification process is a bit annoying. Why should I bother to actually submit spam to spamcop and have it verified? If I just delete it, that will take less time...
The human being isn't really made to do repeating things. This gets quickly boring and hence my idea to automate this submission and verfication process.
I'm not yet an advanced linux user and/or coder or sysadmin or whatever. I just share all the knowledge I have. In order for this tutorial to run you will need several things:
1.) Maildir structure for your email (I'm just looping in the directory through all the emails hence maildb won't work with this)
2.) Be able to setup cron jobs (otherwise you can't automate a thing)
3.) Be able to run shell script (in Bash)
4.) Be able to run PHP script from shell (I do that with Lynx)
5.) A small programm called mime-construct. You can install it on Debian like this:
apt-get install mime-construct
6.) The Snoopy Class (a PHP class used for submitting webforms and other things)
7.) Eventually you need to have mailfiltering capabilites, especially if you use catch-all email addresses (e.g. procmail).
First of all, you will need to create an account at Spamcop. This is, of course, free of charge.
Then you need to create a folder where you put all your spam into. On my system I just call it "Spam". (Since Maildir is a prerequisite the important folder is the cur folder under the spam folder, in my case /home/mail/web4p1/Maildir/.Spam/cur/).
Spam forward script
Now that we have more or less everything together, it's time for the first script. We put now all the spam that come through our RBLs and Spamassassin into our "Spam" folder where the spams are actually put into the subfolder "cur".
All in all, this is a very simple script and just two things need to be adjusted:
1.) The PATH variable needs to point to your spam/cur folder
2.) The EMAIL variable needs to be set to the one you have received upon signing up at spamcop
In my script I also teach Spamassassin about those spams and then I delete them. You may want to handle them differently. Important however is, that once you have submitted those emails to spamcop, that you don't resubmit them again. Either delete them or move them in some other place. If you don't have spamassassin enabled to make use of Bayes then also remove the Spamassassin learning line.
Spam verification script
As said in the prerequisites we also need a folder where the verification emails from spamcop go to. This can be either a complete new email account or some folder combined with some email filtering. I have opted for the second option and I use procmail to filter my incoming email:
:0:Now that we have another folder for the verification emails we need to filter out the unique ID that is contained in them. I have created this little script to get the whole url:
vs.sh (verify spam script)
Again, quite a simple script. All it does is go to the path given, loop through all the emails contained there, filtering out the line with the ID and passing that information to a PHP script (which will then do the actual form submission).
1.) The PATH variable needs to point to your spam/cur folder.
2.) The URL variable needs to be set to your weblocation of the script.
Spamcop form submission script
Well, so far we have forwarded all spam emails to spamcop, received their verifcation emails containing the ID for the form submisson and sent that data to a PHP script.
<?phpI have added quite a few comments for you to understand the logic.
One thing that might be changed is the line
$submit_vars["send".$i] = "on";This line may be removed but then this line here:
$offender = array("type", "master", "info", "sc_comment", "comment");Needs to be altered to:
$offender = array("type", "master", "info", "sc_comment", "comment", "on");In the unaltered version you tell spamcop to send a mail to every entry found in the headers, while the altered version uses the recommendations by spamcop (this is probably the safer method).
Cron SetupWe now have everything together, however we would be required to execute the two bash scripts manually (fe.sh and vs.sh). Well, this tutorial is about on how to automate this so the last step is now to create cronjobs which do the job for us.
I personally prefer creating text files and adding them then to crontab. In order to do so, simply create a file containing this:
cron.txt (cron command file)
0,20,40 * * * * /bin/sh /PATH/TO/fe.shOf course you have to replace /PATH/TO with your own path to where the files are. In my case it's this:
0,20,40 * * * * /bin/sh /home/mail/web4p1/fe.sh > /home/mail/web4p1/output1.txt
Note: I have added there also an output in order to see whether the crons and scripts run fine. Once you are satisfied, just delete the > .... stuff from the cron text file
So, now we have created the cron text file but how do we add it as cronjob? The answer is straight forward:
crontab -uUSER cron.txt
Just replace USER with the user you want to run the cronjob under or just leave -uUSER away if you are logged in as this user or as root and want to have it run as root (not recommended!!!)
Well, that's it.
1.) You can download a copy of the scripts from the forum.
2.) Don't forget to chown and chmod the files correctly (I have made the shell script executing for the user - however I'm not sure if that is required).
3.) You only need 1 vs.sh script if you keep using the same spamcop submission email. All that is required in order to make use of the auto-submission is creating a "Spam" folder in each email account and have the fe.sh script run on it.
4.) I set cron to run every 20 minutes... very likely you want to change that to once an hour... however speed of submission is crucial. The faster you submit and verify spam the sooner it will appear in the spamcop RBL. Because of that and because I'm almost non-stop online when awake during the day (as it is my job) I set cron to run every 20 minutes.
If you have improvements and suggestions, let me know :)