DSPAM tutorial

Discussion in 'Suggest HOWTO' started by bernd, May 26, 2005.

  1. bernd

    bernd New Member

  2. Bailx

    Bailx New Member

    I'm going to be working on getting dspam to work in place of spamassassin for this tutorial http://www.howtoforge.com/book/print/181

    i'll let you know if i'm successful... i'd be open to any suggestions in the meantime... as it might be a while before i'm to that stage, i'm still working on being "satisfied" with ubuntu smp kernel... as I'm coming off of using RHEL v4 (64-bit)... also that's a debian howto... ;)
     
  3. falko

    falko Super Moderator

    I worked on exactly the same a few weeks ago (got it working), but there were some issues to which I couldn't find a solution, that's why I postponed such a tutorial:
    - In such a setup DSpam scans also outgoing emails and puts the DSpam ID into the mail body which is not good if there's a spam scanner at the other end which doesn't like unique IDs in the mail body...
    - When I sent mails to DSpam for learning, but with a faked DSpam ID in the mail body, the DSpam daemon got a fatal error and broke down... :( :( (and I could reproduce that behaviour)
    - In the web interface you can re-learn messages only on the first page of the message list...
     
  4. Bailx

    Bailx New Member

    Did you try all 3 methods of integration? Just curious because if not... perhaps one of the other methods would do the trick.

    As a delivery proxy:
    The default approach integrates DSPAM directly with the mail server and
    filters spam as mail comes in. Please see the appropriate instructions
    in doc/ pertaining to your MTA.

    As a POP3 proxy:
    This alternative approach implements a POP3 proxy where users
    connect to the proxy to check their email, and email is filtered when
    being downloaded. The POP3 proxy is a much easier approach, as it
    requires much less integration work with the mail server (and is ideal
    for implementing DSPAM on Exchange, etcetera). Please see the file
    doc/pop3filter.txt.

    As an SMTP Relay:
    DSPAM can be configured as an SMTP relay, a.k.a appliance. You
    can set it up to sit in front of your real mail server and then point
    your MX records at it. DSPAM will then pass along the good mail to
    your real SMTP server. See doc/relay.txt for more information. The
    example provided uses Postfix and MySQL.
     
  5. falko

    falko Super Moderator

    I integrated DSpam into Postfix as a content filter:

    Code:
    [Postfix] (LMTP) -> [DSPAM]                     [Postfix] -> { Delivery }
                           |___ (SMTP Reinjection) ____|
    because then I don't have to care about any setuid permissions.
     
  6. his_royal_evilness

    his_royal_evilness New Member

    Could you provide a howto Falko? :)
     
  7. falko

    falko Super Moderator

    Sure - if I get it to work the way I want to... :D I don't write tutorials about half-cooked things...
     
  8. his_royal_evilness

    his_royal_evilness New Member

    Thnx in advance , how far are you with the setup? :)
     
  9. falko

    falko Super Moderator

    I have postponed this because I had difficulties for which I couldn't find a solution then... :(
     
  10. his_royal_evilness

    his_royal_evilness New Member

    What kind of ?
    Perhaps we could work together on this one? :)
     
  11. falko

    falko Super Moderator

    I mentioned them earlier in this thread.
     
  12. his_royal_evilness

    his_royal_evilness New Member

    let work together on this one :)
    If you could provide me with ur setup (dspam configure options / postfix configuration) it would be very nice to test it at my server :)
     
  13. falko

    falko Super Moderator

    These are my notes about what I've done so far (this setup is based on a basic Debian Sarge installation):

    Code:
    groupadd dspam
    useradd -s /bin/false -m -c "DSpam" -g dspam dspam
    usermod -G mail dspam
    
    cd /tmp
    wget http://dspam.nuclearelephant.com/sources/dspam-3.6.2.tar.gz
    tar xvfz dspam-3.6.2.tar.gz
    cd dspam-3.6.2
    ./configure --sysconfdir=/etc --with-dspam-home=/var/dspam --enable-domain-scale --with-storage-driver=mysql_drv,hash_drv --enable-daemon --with-mysql-includes=/usr/include/mysql --with-mysql-libraries=/usr/lib --enable-virtual-users --enable-preferences-extension --enable-clamav
    make
    make install
    
    cp -r txt/ /var/dspam
    
    mysqladmin -u root -p create dspam
    mysql -u root -p
    GRANT SELECT, INSERT, UPDATE, DELETE ON dspam.* TO 'dspam_admin'@'localhost' IDENTIFIED BY 'dspam_admin_password';
    GRANT SELECT, INSERT, UPDATE, DELETE ON dspam.* TO 'dspam_admin'@'localhost.localdomain' IDENTIFIED BY 'dspam_admin_password';
    FLUSH PRIVILEGES;
    quit;
    mysql -u root -p<password> dspam < src/tools.mysql_drv/mysql_objects-speed.sql
    mysql -u root -p<password> dspam < src/tools.mysql_drv/virtual_users.sql
    
    mkdir /var/dspam/sql
    cp src/tools.mysql_drv/purge.sql /var/dspam/sql/
    
    /var/dspam/sql/purge.sgl:
    =========================
    delete from dspam_token_data
      where @a-to_days(last_hit) > 90;   entfernen ---->
    
    # $Id: purge.sql,v 1.4 2005/05/04 20:14:37 jonz Exp $
    set @a=to_days(current_date());
    delete from dspam_token_data
      where (innocent_hits*2) + spam_hits < 5
      and @a-to_days(last_hit) > 60;
    delete from dspam_token_data
      where innocent_hits = 1 and spam_hits = 0
      and @a-to_days(last_hit) > 15;
    delete from dspam_token_data
      where innocent_hits = 0 and spam_hits = 1
      and @a-to_days(last_hit) > 15;
    delete from dspam_signature_data
      where @a-14 > to_days(created_on);
    
    
    
    rm -f /etc/alternatives/editor
    ln -s /usr/bin/vi /etc/alternatives/editor
    
    crontab -e
    
    0 0 * * * mysql -u root -p<password> dspam < /var/dspam/sql/purge.sgl
    0 0 * * * dspam_logrotate -a 30 /var/dspam/system.log `find /var/dspam/data -name "*.log"`
    
    /etc/postfix/master.cf:
    =======================
    smtp      inet  n       -       -       -       -        smtpd
                -o content_filter=lmtp:unix:/tmp/dspam.sock
    
    localhost:10026 inet  n -       -       -       -        smtpd
      -o content_filter=
      -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
      -o smtpd_helo_restrictions=
      -o smtpd_client_restrictions=
      -o smtpd_sender_restrictions=
      -o smtpd_recipient_restrictions=permit_mynetworks,reject
      -o mynetworks=127.0.0.0/8
      -o smtpd_authorized_xforward_hosts=127.0.0.0/8
    
    
    
    /etc/dspam.conf:
    ================
    #TrustedDeliveryAgent
    DeliveryHost        127.0.0.1
    DeliveryPort        10026
    DeliveryIdent       localhost
    DeliveryProto       SMTP
    
    Trust root
    Trust mail
    Trust dspam
    
    MySQLServer    127.0.0.1
    MySQLPort      3306
    MySQLUser      dspam_admin
    MySQLPass      dspam_admin_password
    MySQLDb        dspam
    MySQLCompress  true
    
    MySQLConnectionCache        10
    
    MySQLVirtualTable          dspam_virtual_uids
    MySQLVirtualUIDField       uid
    MySQLVirtualUsernameField  username
    
    MySQLUIDInSignature    on
    
    Notifications        on
    
    ParseToHeaders on
    ChangeModeOnParse on
    ChangeUserOnParse full
    
    ServerMode                auto
    ServerParameters        "--deliver=innocent"
    ServerIdent             "localhost.localdomain"
    ServerPID               /var/run/dspam.pid
    ServerDomainSocketPath        "/tmp/dspam.sock"
    
    
    /etc/init.d/postfix restart
    
    dspam_admin ch pref default trainingMode TEFT
    dspam_admin ch pref default spamAction quarantine
    dspam_admin ch pref default spamSubject "[SPAM]"
    dspam_admin ch pref default enableWhitelist on
    dspam_admin ch pref default showFactors off
    
    #dspam --daemon &
    /etc/init.d/dspam anlegen
    chmod 755 /etc/init.d/dspam
    update-rc.d dspam defaults
    /etc/init.d/dspam start
    
    apt-get install apache-dev
    apt-get install libgd-gd2-perl libgd2 libgd2-dev libgd2-xpm libgd2-xpm-dev
    apt-get install libgd-graph3d-perl libgd-graph-perl libgd-text-perl
    mkdir -p /var/www/dspam/web
    mkdir /var/www/dspam/cgi-bin
    cd /tmp
    wget http://mesh.dl.sourceforge.net/sourceforge/modauthmysql/mod_auth_mysql-3.0.0.tar.gz
    tar xvfz mod_auth_mysql-3.0.0.tar.gz
    cd mod_auth_mysql-3.0.0
    apxs -c -I/usr/include/mysql -lmysqlclient -lm -lz mod_auth_mysql.c
    apxs -i mod_auth_mysql.so
    
    /etc/apache/modules.conf:
    =======================
    LoadModule mysql_auth_module /usr/lib/apache/1.3/mod_auth_mysql.so
    
    /etc/init.d/apache restart
    
    cd /tmp/dspam-3.6.2/webui
    cp -pfr cgi-bin/* /var/www/dspam/cgi-bin
    cp -pfr htdocs/* /var/www/dspam/web/
    cd /var/www/dspam
    chown -R dspam:dspam cgi-bin/
    chown -R dspam:dspam web/
    
    /var/www/dspam/cgi-bin/configure.pl:
    ====================================
    $CONFIG{'WEB_ROOT'}     = ""; # URL location of included htdocs/ files
    $CONFIG{'LOCAL_DOMAIN'} = "test.de";
    $CONFIG{'HISTORY_SIZE'} = 799;          # Number of items in history
    $CONFIG{'HISTORY_PER_PAGE'} = 800;
    
    
    /etc/apache/httpd.conf:
    =======================
    NameVirtualHost 192.168.0.101
    
    <VirtualHost 192.168.0.101>
    User dspam
    Group dspam
    DocumentRoot /var/www/dspam/web
    DirectoryIndex index.html index.htm index.php index.php4 index.php3 index.shtml index.cgi index.pl index.jsp Default.htm default.htm
    ScriptAlias /cgi-bin/ /var/www/dspam/cgi-bin/
    AddHandler cgi-script .cgi
    AddHandler cgi-script .pl
    <Directory /var/www/dspam/web>
        Options +Includes -Indexes
        AllowOverride None
        AllowOverride Indexes AuthConfig Limit FileInfo
        Order allow,deny
        Allow from all
        <Files ~ "^\.ht">
        Deny from all
        </Files>
    </Directory>
    <Directory "/var/www/dspam/cgi-bin">
    Options +ExecCGI
      AuthName "DSPAM Quarantine Area"
      AuthType Basic
      AuthMySQLEnable on
      AuthMySQLHost 127.0.0.1
      AuthMySQLPort 3306
      AuthMySQLUser mail_admin
      AuthMySQLPassword mail_admin_password
      AuthMySQLDB mail
      AuthMySQLUserTable users
      AuthMySQlNameField email
      AuthMySQLPasswordField password
      AuthMySQLPwEncryption crypt
      Require valid-user
      Order Deny,allow
      Allow from all
    </Directory>
    </VirtualHost>
    
    mv /usr/lib/apache/suexec.disabled /usr/lib/apache/suexec
    /etc/init.d/apache restart
    
    
    Retrain:
    --------
    
    /usr/local/bin/dspam-retrain:
    =============================
    #! /usr/bin/perl
    
    # Get arguments
    $class  = $ARGV[0] || die; shift;
    $sender = $ARGV[0] || die; shift;
    $recip  = $ARGV[0] || die; shift;
    
    if ($recip =~ /^(spam|ham)-(\w+)@/) {
        # username is part of the recipient
        $user = $2;
    } elsif ($sender =~ /^(\w+)@/) {
        # username is in the sender
        $user = $1;
    } else {
        print "Can't determine user\n";
        exit 75;                    # EX_TEMPFAIL
    }
    
    # Pull out DSPAM signatures and send them to the dspam program
    while (<>) {
        if ((! $subj) && (/^Subject: /)) {
            $subj = $_;
        } elsif (/(!DSPAM:[a-f0-9]+!)/) {
            open(F, "|/usr/local/bin/dspam --source=error --class=$class --user $user");
            print F "$subj\n$1\n";
            close(F);
        } elsif (/(X-DSPAM-Signature: [a-f0-9]+)/) {
            open(F, "|/usr/local/bin/dspam --source=error --class=$class --user $user");
            print F "$subj\n$1\n";
            close(F);
        }
    }
    
    
    
    
    chmod 755 /usr/local/bin/dspam-retrain
    
    /etc/postfix/master.cf:
    =======================
    dspam-retrain   unix    -       n       n       -       10      pipe
      flags=Ru user=dspam argv=/usr/local/bin/dspam-retrain $nexthop $sender $recipient
    
    
    transport-Tabelle:
    ===================
    spam@test.de    dspam-retrain:spam
    ham@test.de     dspam-retrain:innocent
    
    INSERT INTO transport(domain, transport) VALUES ('spam@test.de', 'dspam-retrain:spam');
    INSERT INTO transport(domain, transport) VALUES ('ham@test.de', 'dspam-retrain:innocent');
    
    INSERT INTO `users` (`email`, `password`, `quota`) VALUES ('spam@test.de', ENCRYPT('secret'), 10485760);
    INSERT INTO `users` (`email`, `password`, `quota`) VALUES ('ham@test.de', ENCRYPT('secret'), 10485760);
    
    /etc/postfix/main.cf:
    =====================
    postconf -e 'local_recipient_maps = proxy:unix:passwd.byname $alias_maps $transport_maps'
    #recipient_delimiter = -
    
    /etc/init.d/postfix restart
    
    
    ClamAV:
    =======
    apt-get install libgmp3 libgmp3-dev
    cd /tmp
    groupadd clamav
    useradd -g clamav -s /bin/false -c "Clam AntiVirus" clamav
    wget http://mesh.dl.sourceforge.net/sourceforge/clamav/clamav-0.87.1.tar.gz
    tar xvfz clamav-0.87.1.tar.gz
    cd clamav-0.87.1
    ./configure --sysconfdir=/etc
    make
    make install
    
    /etc/clamd.conf bearbeiten
    /etc/freshclam.conf bearbeiten
    /etc/init.d/clamd anlegen
    chmod 755 /etc/init.d/clamd
    update-rc.d clamd defaults
    /etc/init.d/clamd start
    
    /etc/dspam.conf:
    ================
    #
    # Virus Checking: If you are running clamd, DSPAM can perform stream-based
    # virus checking using TCP. Uncomment the values below to enable virus
    # checking.
    #
    # ClamAVResponse: reject (reject or drop the message with a permanent failure)
    #                 accept (accept the message and quietly drop the message)
    #                 spam   (treat as spam and quarantine/tag/whatever)
    #
    ClamAVPort        3310
    ClamAVHost        127.0.0.1
    ClamAVResponse accept
    
    
    /etc/init.d/dspam restart
    
    
    /usr/local/bin/dspam --user ft@test.de --class=spam --source=error --signature=1,4399a03667787982714935
     
  14. his_royal_evilness

    his_royal_evilness New Member

    Thanks , i'll have a look at this and will see if i can get it to work/enhance it
    ( Think of automatic spam-user@domain parsing with retraining spam )

    Also : Could you translate the german words in English for me as German isn't my best language *looks nicely*
     
    Last edited: Jan 29, 2006
  15. his_royal_evilness

    his_royal_evilness New Member

    AuthMySQLUser mail_admin
    AuthMySQLPassword mail_admin_password

    What does that mean? on what database?
     
  16. till

    till Super Moderator

    I guess thats the login to the DSPAM mySQL database. Most DSPAM installations run on mySQL because it has the best performance of the DSPAM storage backends.
     
  17. his_royal_evilness

    his_royal_evilness New Member

    Mkay , i want users to authenticate with their user@domain adress at the Webinterface of dspam , any idea on how to implement that?
     
  18. brent.stephens

    brent.stephens New Member

    I am what I consider to be successfully running DSPAM "in front" of ISPConfig at this point. I am using the latest DSPAM CVS, which is the basis for what will be the 3.8 release (current release is 3.6.8).

    I will create a howto with my internal wiki and once its finished will share it with you guys for editing and suggestions.
     
  19. falko

    falko Super Moderator

    That would be great! :)
     
  20. brent.stephens

    brent.stephens New Member

    Well its not on my internal wiki, since I love to share the DSPAM love ;)

    This is my first real draft of the process I used to set up DSPAM. After a couple weeks now I am still very happy with it. Being that it is a first draft, it is basically a regurgitation of thought and config files. Think of it as a barebones kit for those who are comfortable with moderately technical Linux stuff :)

    Please forgive me in advance for not being as unbelievably thorough as falko and till.

    http://dspamwiki.expass.de/Installation/Postfix/DspamWithIspConfig
     

Share This Page