DSPAM tutorial

[falko]

Originally Posted by his_royal_evilness

What kind of ?

I mentioned them earlier in this thread.

[his_royal_evilness]

let work together on this one
If you could provide me with ur setup (dspam configure options / postfix configuration) it would be very nice to test it at my server

[falko]

These are my notes about what I've done so far (this setup is based on a basic Debian Sarge installation):

groupadd dspam
useradd -s /bin/false -m -c "DSpam" -g dspam dspam
usermod -G mail dspam

cd /tmp
wget http://dspam.nuclearelephant.com/sources/dspam-3.6.2.tar.gz
tar xvfz dspam-3.6.2.tar.gz
cd dspam-3.6.2
./configure --sysconfdir=/etc --with-dspam-home=/var/dspam --enable-domain-scale --with-storage-driver=mysql_drv,hash_drv --enable-daemon --with-mysql-includes=/usr/include/mysql --with-mysql-libraries=/usr/lib --enable-virtual-users --enable-preferences-extension --enable-clamav
make
make install

cp -r txt/ /var/dspam

mysqladmin -u root -p create dspam
mysql -u root -p
GRANT SELECT, INSERT, UPDATE, DELETE ON dspam.* TO 'dspam_admin'@'localhost' IDENTIFIED BY 'dspam_admin_password';
GRANT SELECT, INSERT, UPDATE, DELETE ON dspam.* TO 'dspam_admin'@'localhost.localdomain' IDENTIFIED BY 'dspam_admin_password';
FLUSH PRIVILEGES;
quit;
mysql -u root -p<password> dspam < src/tools.mysql_drv/mysql_objects-speed.sql
mysql -u root -p<password> dspam < src/tools.mysql_drv/virtual_users.sql

mkdir /var/dspam/sql
cp src/tools.mysql_drv/purge.sql /var/dspam/sql/

/var/dspam/sql/purge.sgl:
=========================
delete from dspam_token_data
  where @a-to_days(last_hit) > 90;   entfernen ---->

# $Id: purge.sql,v 1.4 2005/05/04 20:14:37 jonz Exp $
set @a=to_days(current_date());
delete from dspam_token_data
  where (innocent_hits*2) + spam_hits < 5
  and @a-to_days(last_hit) > 60;
delete from dspam_token_data
  where innocent_hits = 1 and spam_hits = 0
  and @a-to_days(last_hit) > 15;
delete from dspam_token_data
  where innocent_hits = 0 and spam_hits = 1
  and @a-to_days(last_hit) > 15;
delete from dspam_signature_data
  where @a-14 > to_days(created_on);



rm -f /etc/alternatives/editor
ln -s /usr/bin/vi /etc/alternatives/editor

crontab -e

0 0 * * * mysql -u root -p<password> dspam < /var/dspam/sql/purge.sgl
0 0 * * * dspam_logrotate -a 30 /var/dspam/system.log `find /var/dspam/data -name "*.log"`

/etc/postfix/master.cf:
=======================
smtp      inet  n       -       -       -       -        smtpd
            -o content_filter=lmtp:unix:/tmp/dspam.sock

localhost:10026 inet  n -       -       -       -        smtpd
  -o content_filter=
  -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
  -o smtpd_helo_restrictions=
  -o smtpd_client_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks=127.0.0.0/8
  -o smtpd_authorized_xforward_hosts=127.0.0.0/8



/etc/dspam.conf:
================
#TrustedDeliveryAgent
DeliveryHost        127.0.0.1
DeliveryPort        10026
DeliveryIdent       localhost
DeliveryProto       SMTP

Trust root
Trust mail
Trust dspam

MySQLServer    127.0.0.1
MySQLPort      3306
MySQLUser      dspam_admin
MySQLPass      dspam_admin_password
MySQLDb        dspam
MySQLCompress  true

MySQLConnectionCache        10

MySQLVirtualTable          dspam_virtual_uids
MySQLVirtualUIDField       uid
MySQLVirtualUsernameField  username

MySQLUIDInSignature    on

Notifications        on

ParseToHeaders on
ChangeModeOnParse on
ChangeUserOnParse full

ServerMode                auto
ServerParameters        "--deliver=innocent"
ServerIdent             "localhost.localdomain"
ServerPID               /var/run/dspam.pid
ServerDomainSocketPath        "/tmp/dspam.sock"


/etc/init.d/postfix restart

dspam_admin ch pref default trainingMode TEFT
dspam_admin ch pref default spamAction quarantine
dspam_admin ch pref default spamSubject "[SPAM]"
dspam_admin ch pref default enableWhitelist on
dspam_admin ch pref default showFactors off

#dspam --daemon &
/etc/init.d/dspam anlegen
chmod 755 /etc/init.d/dspam
update-rc.d dspam defaults
/etc/init.d/dspam start

apt-get install apache-dev
apt-get install libgd-gd2-perl libgd2 libgd2-dev libgd2-xpm libgd2-xpm-dev
apt-get install libgd-graph3d-perl libgd-graph-perl libgd-text-perl
mkdir -p /var/www/dspam/web
mkdir /var/www/dspam/cgi-bin
cd /tmp
wget http://mesh.dl.sourceforge.net/sourceforge/modauthmysql/mod_auth_mysql-3.0.0.tar.gz
tar xvfz mod_auth_mysql-3.0.0.tar.gz
cd mod_auth_mysql-3.0.0
apxs -c -I/usr/include/mysql -lmysqlclient -lm -lz mod_auth_mysql.c
apxs -i mod_auth_mysql.so

/etc/apache/modules.conf:
=======================
LoadModule mysql_auth_module /usr/lib/apache/1.3/mod_auth_mysql.so

/etc/init.d/apache restart

cd /tmp/dspam-3.6.2/webui
cp -pfr cgi-bin/* /var/www/dspam/cgi-bin
cp -pfr htdocs/* /var/www/dspam/web/
cd /var/www/dspam
chown -R dspam:dspam cgi-bin/
chown -R dspam:dspam web/

/var/www/dspam/cgi-bin/configure.pl:
====================================
$CONFIG{'WEB_ROOT'}     = ""; # URL location of included htdocs/ files
$CONFIG{'LOCAL_DOMAIN'} = "test.de";
$CONFIG{'HISTORY_SIZE'} = 799;          # Number of items in history
$CONFIG{'HISTORY_PER_PAGE'} = 800;


/etc/apache/httpd.conf:
=======================
NameVirtualHost 192.168.0.101

<VirtualHost 192.168.0.101>
User dspam
Group dspam
DocumentRoot /var/www/dspam/web
DirectoryIndex index.html index.htm index.php index.php4 index.php3 index.shtml index.cgi index.pl index.jsp Default.htm default.htm
ScriptAlias /cgi-bin/ /var/www/dspam/cgi-bin/
AddHandler cgi-script .cgi
AddHandler cgi-script .pl
<Directory /var/www/dspam/web>
    Options +Includes -Indexes
    AllowOverride None
    AllowOverride Indexes AuthConfig Limit FileInfo
    Order allow,deny
    Allow from all
    <Files ~ "^\.ht">
    Deny from all
    </Files>
</Directory>
<Directory "/var/www/dspam/cgi-bin">
Options +ExecCGI
  AuthName "DSPAM Quarantine Area"
  AuthType Basic
  AuthMySQLEnable on
  AuthMySQLHost 127.0.0.1
  AuthMySQLPort 3306
  AuthMySQLUser mail_admin
  AuthMySQLPassword mail_admin_password
  AuthMySQLDB mail
  AuthMySQLUserTable users
  AuthMySQlNameField email
  AuthMySQLPasswordField password
  AuthMySQLPwEncryption crypt
  Require valid-user
  Order Deny,allow
  Allow from all
</Directory>
</VirtualHost>

mv /usr/lib/apache/suexec.disabled /usr/lib/apache/suexec
/etc/init.d/apache restart


Retrain:
--------

/usr/local/bin/dspam-retrain:
=============================
#! /usr/bin/perl

# Get arguments
$class  = $ARGV[0] || die; shift;
$sender = $ARGV[0] || die; shift;
$recip  = $ARGV[0] || die; shift;

if ($recip =~ /^(spam|ham)-(\w+)@/) {
    # username is part of the recipient
    $user = $2;
} elsif ($sender =~ /^(\w+)@/) {
    # username is in the sender
    $user = $1;
} else {
    print "Can't determine user\n";
    exit 75;                    # EX_TEMPFAIL
}

# Pull out DSPAM signatures and send them to the dspam program
while (<>) {
    if ((! $subj) && (/^Subject: /)) {
        $subj = $_;
    } elsif (/(!DSPAM:[a-f0-9]+!)/) {
        open(F, "|/usr/local/bin/dspam --source=error --class=$class --user $user");
        print F "$subj\n$1\n";
        close(F);
    } elsif (/(X-DSPAM-Signature: [a-f0-9]+)/) {
        open(F, "|/usr/local/bin/dspam --source=error --class=$class --user $user");
        print F "$subj\n$1\n";
        close(F);
    }
}




chmod 755 /usr/local/bin/dspam-retrain

/etc/postfix/master.cf:
=======================
dspam-retrain   unix    -       n       n       -       10      pipe
  flags=Ru user=dspam argv=/usr/local/bin/dspam-retrain $nexthop $sender $recipient


transport-Tabelle:
===================
spam@test.de    dspam-retrain:spam
ham@test.de     dspam-retrain:innocent

INSERT INTO transport(domain, transport) VALUES ('spam@test.de', 'dspam-retrain:spam');
INSERT INTO transport(domain, transport) VALUES ('ham@test.de', 'dspam-retrain:innocent');

INSERT INTO `users` (`email`, `password`, `quota`) VALUES ('spam@test.de', ENCRYPT('secret'), 10485760);
INSERT INTO `users` (`email`, `password`, `quota`) VALUES ('ham@test.de', ENCRYPT('secret'), 10485760);

/etc/postfix/main.cf:
=====================
postconf -e 'local_recipient_maps = proxy:unix:passwd.byname $alias_maps $transport_maps'
#recipient_delimiter = -

/etc/init.d/postfix restart


ClamAV:
=======
apt-get install libgmp3 libgmp3-dev
cd /tmp
groupadd clamav
useradd -g clamav -s /bin/false -c "Clam AntiVirus" clamav
wget http://mesh.dl.sourceforge.net/sourceforge/clamav/clamav-0.87.1.tar.gz
tar xvfz clamav-0.87.1.tar.gz
cd clamav-0.87.1
./configure --sysconfdir=/etc
make
make install

/etc/clamd.conf bearbeiten
/etc/freshclam.conf bearbeiten
/etc/init.d/clamd anlegen
chmod 755 /etc/init.d/clamd
update-rc.d clamd defaults
/etc/init.d/clamd start

/etc/dspam.conf:
================
#
# Virus Checking: If you are running clamd, DSPAM can perform stream-based
# virus checking using TCP. Uncomment the values below to enable virus
# checking.
#
# ClamAVResponse: reject (reject or drop the message with a permanent failure)
#                 accept (accept the message and quietly drop the message)
#                 spam   (treat as spam and quarantine/tag/whatever)
#
ClamAVPort        3310
ClamAVHost        127.0.0.1
ClamAVResponse accept


/etc/init.d/dspam restart


/usr/local/bin/dspam --user ft@test.de --class=spam --source=error --signature=1,4399a03667787982714935

[his_royal_evilness]

Thanks , i'll have a look at this and will see if i can get it to work/enhance it
( Think of automatic spam-user@domain parsing with retraining spam )

Also : Could you translate the german words in English for me as German isn't my best language *looks nicely*

[his_royal_evilness]

AuthMySQLUser mail_admin
AuthMySQLPassword mail_admin_password

What does that mean? on what database?

[till]

I guess thats the login to the DSPAM mySQL database. Most DSPAM installations run on mySQL because it has the best performance of the DSPAM storage backends.

[his_royal_evilness]

Mkay , i want users to authenticate with their user@domain adress at the Webinterface of dspam , any idea on how to implement that?

[brent.stephens]

I am what I consider to be successfully running DSPAM "in front" of ISPConfig at this point. I am using the latest DSPAM CVS, which is the basis for what will be the 3.8 release (current release is 3.6.8).

I will create a howto with my internal wiki and once its finished will share it with you guys for editing and suggestions.

[falko]

Originally Posted by brent.stephens

I will create a howto with my internal wiki and once its finished will share it with you guys for editing and suggestions.

That would be great!

[brent.stephens]

Well its not on my internal wiki, since I love to share the DSPAM love

This is my first real draft of the process I used to set up DSPAM. After a couple weeks now I am still very happy with it. Being that it is a first draft, it is basically a regurgitation of thought and config files. Think of it as a barebones kit for those who are comfortable with moderately technical Linux stuff

Please forgive me in advance for not being as unbelievably thorough as falko and till.

http://dspamwiki.expass.de/Installat...mWithIspConfig