Custom Spamassassin rules/score/filter

Discussion in 'Installation/Configuration' started by Ante-Florian, Sep 6, 2019.

  1. Ante-Florian

    Ante-Florian New Member

    Hi there!
    I'm currently migrating a server from "do it all over cli" to ISPConfig 3.
    Now, i'm struggling with my Spamassassin config.
    What i want
    I want to have two (or more) Spamfilter Policy's who affect different:
    The one have some custom Spamassassin Modules loaded (RelayCountry) and Custom Scores for different modules. This is the default rule added to the Domain with an score of 5. The other one is a Mailbox-based Policy and has a Score of 10.
    Steps to reproduce
    1) In ISPconfig i create a new Domain under EMail > Domain (no client was created before!)
    2) Create a new Mailbox
    3) Create a new Spamfilter Policy, named "Custom"
    4) Assign the Custom Spamfilter policy to the Domain. I want that this Policy is execute on all mailboxes of the Domain - also alias.
    5) Tweek some settings in the custom Policy settings of ISPConfig3:
    Policy
    • Policy Name: Custom
    • Virus lover: No
    • SPAM lover:
    • Banned files lover: No
    • Bad header lover: No
    • Bypass virus checks: No
    • Bypass banned checks: No
    • Bypass header checks: No

    Quarantine
    Only for Debug purpose
    • Forward virus to email: (links not allowed)
    • Forward spam to email: (links not allowed)
    • Forward banned to email: (links not allowed)
    • Forward bad header to email: (links not allowed)

    Tag-Level
    • SPAM tag level: -1 //Only for Debug purpose, want to print the tests in Mailheader. Normal 5
    • SPAM tag2 level: 10
    • SPAM kill level: 50
    • SPAM dsn cutoff level: 45
    • SPAM quarantine cutoff level: 0
    • SPAM modifies subject: No
    • SPAM subject tag: EMPTY
    • SPAM subject tag2: [SPAM]

    Other
    • Addr. extension virus: EMPTY
    • Addr. extension SPAM: EMPTY
    • Addr. extension banned: EMPTY
    • Addr extension bad header: EMPTY
    • Warn virus recip.: No
    • Warn banned recip.: No
    • Warn bad header recip.: No
    • Newvirus admin: EMPTY
    • Virus admin:EMPTY
    • Banned admin: EMPTY
    • Bad header admin
    • SPAM admin:EMPTY
    • Message size limit: 0 // for Debug, normal 100 MB in Bytes
    • Banned rulenames: RMPTY
    This are how i interpret the Settings and adapt to my needs. (links not allowed)

    In the file /etc/spamassassin/local.cf i add this lines:
    Code:
    // to use the sa-learn generated Bayes files
    bayes_path /var/lib/amavis/.spamassassin/bayes 
    Code:
    // Score some Relays different.
    ifplugin Mail::SpamAssassin::plugin::RelayCountry
    add_header all Relay-Country _RELAYCOUNTRY_
    header RELAYCOUNTRY_BAD X-Relay-Countries =~ /(CN|RU|UA|RO|VN)/
    describe RELAYCOUNTRY_BAD Relayed through spammy country at some point
    score RELAYCOUNTRY_BAD 1.0
    
    header RELAYCOUNTRY_GOOD X-Relay-Countries =~ /^(DE|AT|CH)/
    describe RELAYCOUNTRY_GOOD First untrusted GW is DE, AT or CH
    score RELAYCOUNTRY_GOOD -0.5
    endif # Mail::SpamAssassin::plugin::RelayCountry
    
    Code:
    // Custom Scores
    score RCVD_IN_BL_SPAMCOP_NET 0 5.246 0 5.347
    score RCVD_IN_BRBL_LASTEXT 0 5.246 0 5.347
    score URIBL_BLACK 0 5.7 0 5.7
    score URIBL_WS_SURBL 0 2.659 0 2.608
    score URIBL_MW_SURBL 0 2.263 0 2.263
    score URIBL_CR_SURBL 0 2.263 0 2.263
    score URIBL_GREY 0 2.084 0 1.424
    score URIBL_DBL_SPAM    0 4.5 0 4.5
    score URIBL_DBL_PHISH   0 4.5 0 4.5
    score URIBL_DBL_MALWARE 0 4.5 0 4.5
    score URIBL_DBL_BOTNETCC 0 4.5 0 4.5
    score URIBL_DBL_ABUSE_SPAM 0 4.0 0 4.0
    score URIBL_DBL_ABUSE_PHISH 0 4.5 0 4.5
    score URIBL_DBL_ABUSE_MALW  0 4.5 0 4.5
    score URIBL_DBL_ABUSE_BOTCC 0 4.5 0 4.5
    
    Current Status
    The custom settings are not execute. In some delivered Emails I get this header:
    Code:
    X-Spam-Status: No, score=3.366 tagged_above=-1 required=10
        tests=[DKIM_SIGNED=0.1, HTML_IMAGE_ONLY_16=1.048, HTML_MESSAGE=0.001,
        HTML_SHORT_LINK_IMG_2=0.259, SPF_PASS=-0.001, T_DKIM_INVALID=0.01,
        URIBL_ABUSE_SURBL=1.948, URIBL_BLOCKED=0.001]
    But if i scan it over the cli with:
    Code:
    [email protected] ~ spamassassin -t -D < spam-message
    ...
    X-Spam-Status: Yes, score=17.2 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED,
       HTML_IMAGE_ONLY_16,HTML_MESSAGE,HTML_SHORT_LINK_IMG_2,RCVD_IN_SBL_CSS,
       SPF_HELO_NONE,URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM
       autolearn=no autolearn_force=no version=3.4.2
    X-Spam-Relay-Country: HU
    
    ...
    Content analysis details:   (17.2 points, 5.0 required)
    
     pts rule name              description
    ---- ---------------------- --------------------------------------------------
     0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
     1.9 URIBL_ABUSE_SURBL      Contains an URL listed in the ABUSE SURBL
                                blocklist
                                [URIs: digitalnanda.com]
     5.7 URIBL_BLACK            Contains an URL listed in the URIBL blacklist
                                [URIs: digitalnanda.com]
     1.0 HTML_IMAGE_ONLY_16     BODY: HTML: images with 1200-1600 bytes of
                                words
     0.0 HTML_MESSAGE           BODY: HTML included in message
     0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily
                                valid
     3.6 RCVD_IN_SBL_CSS        RBL: Received via a relay in Spamhaus SBL-CSS
                                [xxx.xxx.xxx.xxx listed in zen.spamhaus.org]
     4.5 URIBL_DBL_SPAM         Contains a spam URL listed in the Spamhaus DBL
                                blocklist
                                [URIs: digitalnanda.com]
     0.3 HTML_SHORT_LINK_IMG_2  HTML is very short with a linked image
     0.1 DKIM_INVALID           DKIM or DK signature exists, but is not valid
    
    You can see, the scores a different and the RelayCountry plugin is not loaded. libgeo-ip-perl is installed and the GeoIP.dat.gz + GeoIPv6.dat.gz are uziped to /usr/share/GeoIP/
    Code:
    [email protected] /usr/share/GeoIP # ls -la
    total 34364
    drwxr-xr-x   2 root root     4096 Sep  5 21:04 .
    drwxr-xr-x 169 root root     4096 Sep  3 09:40 ..
    -rw-r--r--   1 root root  4969864 Nov  8  2018 GeoIPASNum.dat
    -rw-r--r--   1 root root 22104026 Nov  8  2018 GeoIPCity.dat
    -rw-r--r--   1 root root  1672893 Nov  8  2018 GeoIP.dat
    -rw-r--r--   1 root root  6426573 Nov  8  2018 GeoIPv6.dat
    
    In /etc/spamassassin/init.pre is RelayCountry loaded
    Code:
    loadplugin Mail::SpamAssassin::Plugin::RelayCountry
    An Email to a alias have this Header. All is set to minimum, but i don't find how this score is generated. Only the tagged_above and required settings are used from the SQL Database (ISPConfig)
    Code:
    X-Spam-Status: No, score=0.302 tagged_above=-1 required=10
    [DKIM_INVALID=0.1, DKIM_SIGNED=0.1,
        HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.1,
        RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001]
    
    Open Questions
    I don't understand how are the Spam Policy passed to amavis-new from ISPConfig. I only find my custom policy in the SQL-Database "spamfilter_policy" and there a no scores or modules to load with spamassassin/amavis.
    In the file /etc/amavis/conf.d/50-user i find some SQL Select statements and Scorings, but they don't match with the used settings. Also sa-update is executed before i start troubleshooting. The /var/log/mail.log and mail.err dont show anything related to my issue.

    1) Where are the scores for the Spam in ISPConfig3 saved?
    2) Where are the loaded modules set and how to change it?

    Now some system information
    htf_report.txt is attached
    Operating System: Debian GNU/Linux 10 (buster)
    Kernel: Linux 4.19.0-5-amd64
    Architecture: x86-64
    Dovecot: 2.3.4.1 (f79e8e7e4)
    Postfix: 3.4.5
    SpamAssassin: 3.4.2
    running on Perl version 5.28.1
    PHP: 7.3.4-2 (cli) (built: Apr 13 2019 19:05:48) ( NTS )
    ISPConfig Version: 3.1.14p2 (update available, but i waiting because change log not show any important changes.
    amavisd-new: 2.11.0 (20160426)

    Thanks, Ante-Florian
     

    Attached Files:

  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Amavis reads the policies directly from ispconfig database. If you search the amavis docs for sql based setup, then you find what ISPConfig is using. And the sql config is in /etc/amavis/conf.d/50-user file.
     
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    Changes for SpamAssassin go into the files in /etc/spamassassin/ and then you restart amavisd to apply them.
     
  4. Ante-Florian

    Ante-Florian New Member

    The changes will not be applied. My changes to /etc/spamassassin/local.cf will not be used.
    Now, i updated ISPConfig to version 3.1.15 and start working with Rspamd.
    Thanks
     
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    I use this config method for more than 10 years on many Debian ISPConfig systems and it works perfectly. Maybe there is a general setup problem with your system then.
     

Share This Page