rspamd blacklist/whitelist support

Discussion in 'ISPConfig 3 Priority Support' started by KoS, Oct 14, 2019.

  1. KoS

    KoS Member HowtoForge Supporter

  2. till

    till Super Moderator Staff Member ISPConfig Developer

    The black- and whitelists are used by rspamd as well.
     
  3. KoS

    KoS Member HowtoForge Supporter

    Thanks @till
    Can you guide me to the right place in the configuration where I can verify the settings? As my tests showed that the blacklist is not active and my users were complaining too :-(
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    You can find the files with:

    ls /etc/rspamd/local.d/users/spamfilter_wblist*
     
  5. KoS

    KoS Member HowtoForge Supporter

    There are no files named like /etc/rspamd/local.d/users/spamfilter_wblist*
    but I have there many files like /etc/rspamd/local.d/users/<user>_<domain>.conf
    In none of these files I see any black/whitelist entries, or a query to the SQL DB or so.

    Just to make sure that we are talking about the same:
    My users have been using until now the function -> Email -> Spamfilter -> Blacklist to block mails for their domain from certain sender mail addresses (or whole sender domains). If I check the users conf file mentioned above for a specifc recipient, or domain, that has blacklist entries, I don't find any reference to it in the conf file. Am I looking and the wrong location or is ISPconfig not creating the proper conf files?

    KoS
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    That's correct, as the spamfilter white- and blacklist is in files with the name I pointed out and not in user_* files.

    I just created a blacklist entry and the file got created successfully.

    Code:
    [email protected]:/# grep -r spammer /etc/rspamd/
    /etc/rspamd/local.d/users/spamfilter_wblist_1.conf:     from = "@spammer.int";
    [email protected]:/# cat /etc/rspamd/local.d/users/spamfilter_wblist_1.conf
    spamfilter_wblist-1 {
            priority = 25;
            from = "@spammer.int";
            rcpt = "@test.tld";
            apply {
                    R_DUMMY = 999.0;
                    actions {
                            reject = 0.2;
                            "add header" = 0.1;
                            greylist = 0.1;
                            "rewrite subject" = 0.1;
                    }
            }
    }
     
  7. KoS

    KoS Member HowtoForge Supporter

    Thank you @till
    I got a step further. I found an inconsistency with the master-slave-sync on my multi-server setup and for that reason the files have not been created. After re-syncing the files have been created, but my test mails are still no captured as spam (R_DUMMY is not being set).

    the spamfilter file looks okay to me (filter all mails from a specific sender for the whole local domain):
    Code:
    spamfilter_wblist-270 {
            priority = 30;
            from = "[email protected]";
            rcpt = "@local.com";
            apply {
                    R_DUMMY = 999.0;
                    actions {
                            reject = 0.2;
                            "add header" = 0.1;
                            greylist = 0.1;
                            "rewrite subject" = 0.1;
                    }
            }
    
    whereas the local user has the following user-specific conf file generated:
    Code:
    ispc_mail_user_15294 {
            priority = 20;
            rcpt = "[email protected]";
    
            apply {
                            CLAM_VIRUS = 1010;
                    JUST_EICAR = 1010;
                            actions {
                                            "rewrite subject" =  6;
                                                            "add header" =  6;
                                            reject = 10;
                                                    greylist = null;
                                            }
            }
    
     
  8. KoS

    KoS Member HowtoForge Supporter

    @till any idea how i could debug this?
     
  9. till

    till Super Moderator Staff Member ISPConfig Developer

    It might be that the local user, as its more specific than the b/w list, has overridden the domain-wide blacklisting. Did you check the rspamd log file to see which scores got applied?
     
  10. KoS

    KoS Member HowtoForge Supporter

    I am not 100% if I understand the log (see below), but i see that my blacklist IS being applied ("apply static settings spamfilter_wblist-271 (id = 3655014795); rcpt,from matched"), but still not reflected in the scoring?

    Code:
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; accept_socket: accepted connection from 127.0.0.1 port 33384, task ptr: 00007F28A425C028
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; rspamd_message_parse: loaded message; id: <[email protected]>; queue-id: <2ADF4705>; size: 785; checksum: <9a6c58deb487ad62ab8417f661cd3a58>
    2019-10-19 00:19:26 #9328(normal) <eaa724>; lua; settings.lua:324: <[email protected]> apply static settings spamfilter_wblist-271 (id = 3655014795); rcpt,from matched
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; spf_plugin_callback: stored record for example.com (0x51fb8b96a373170c) in LRU cache for 1 seconds, 1/2000 elements in the cache
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; rspamd_redis_finalize_process: cannot retreive stat tokens from Redis: skip obtaining bayes tokens for BAYES_SPAM: not enough learns 8; 200 required
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; rspamd_redis_finalize_process: cannot retreive stat tokens from Redis: skip obtaining bayes tokens for BAYES_HAM: not enough learns 3; 200 required
    2019-10-19 00:19:26 #9328(normal) <eaa724>; lua; greylist.lua:298: Score too low - skip greylisting
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; rspamd_stat_check_autolearn: <[email protected]>: autolearn ham for classifier 'bayes' as message's score is negative: -0.31
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; rspamd_task_write_log: id: <[email protected]>, qid: <2ADF4705>, ip: 138.201.250.115, from: <[email protected]>, (default: F (no action): [-0.31/0.10] [R_SPF_ALLOW(-0.20){+a:mail-in.example.com;},MIME_GOOD(-0.10){text/plain;},MX_GOOD(-0.01){cached: mail-in.example.com;},ARC_NA(0.00){},ASN(0.00){asn:24940, ipnet:138.201.0.0/16, country:DE;},DMARC_NA(0.00){example.com;},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},MID_RHS_MATCH_FROM(0.00){},MIME_TRACE(0.00){0:+;},PREVIOUSLY_DELIVERED(0.00){[email protected];},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_TWO(0.00){2;},RCVD_TLS_ALL(0.00){},RCVD_VIA_SMTP_AUTH(0.00){},R_DKIM_NA(0.00){},TO_DN_NONE(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 785, time: 540.634ms, dns req: 36, digest: <9a6c58deb487ad62ab8417f661cd3a58>, rcpts: <[email protected]>, mime_rcpts: <[email protected]>, settings_id: spamfilter_wblist-271
    2019-10-19 00:19:26 #9328(normal) <eaa724>; task; rspamd_protocol_http_reply: regexp statistics: 0 pcre regexps scanned, 4 regexps matched, 184 regexps total, 100 regexps cached, 0B scanned using pcre, 1.01KiB scanned total
    
    below are the rspamd configs from the users-directory
    - the recipient domain (local-recipient.com) is on that mail-server and it has spamfilter set to "normal"
    - the users mailbox spamfilter is set to " - not enabled - "
    - there is one blacklist entry to block any mails from [email protected] to this domain, with priority 10

    I am not sure how to further debug this? i assume i should see the "R_DUMMY" score being applied, which would trigger the reject of the mail?

    Code:
    cat /etc/rspamd/local.d/users/spamfilter_wblist_271.conf
    spamfilter_wblist-271 {
            priority = 30;
            from = "[email protected]";
            rcpt = "@local-recipient.com";
            apply {
                    R_DUMMY = 999.0;
                    actions {
                            reject = 0.2;
                            "add header" = 0.1;
                            greylist = 0.1;
                            "rewrite subject" = 0.1;
                    }
            }
    }
    
    Code:
    cat /etc/rspamd/local.d/users/local-recipient.com.conf
    ispc_mail_forwarding_23322 {
            priority = 18;
            rcpt = "@local-recipient.com";
    
            apply {
                            CLAM_VIRUS = 1010;
                    JUST_EICAR = 1010;
                            actions {
                                            "rewrite subject" =  6;
                                                            "add header" =  6;
                                            reject = 10;
                                                    greylist = null;
                                            }
            }
    
    }
    
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    I haven't written the Rspamd module for ISPConfig and I'm not that familiar with the in-depth details of Rspamd yet, but maybe @Croydon might be able to help you further.
     
  12. KoS

    KoS Member HowtoForge Supporter

    @Croydon Do you have an idea what could be wrong or how I shall best debug it to find the issue? Thanks.
     
    jokerdes likes this.
  13. KoS

    KoS Member HowtoForge Supporter

    @till: have you tested if not just the file gets created, but the blacklist really works when sending mails?
     
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    It had been tested at the time the functions were implemented. But the blacklist is to prevent that a message from a certain sender gets received and not to prevent that a local account can send to someone.
     

Share This Page