Custom Fail2Ban Jail not detecting correct IP to be blocked (Apache)

Discussion in 'Server Operation' started by Nyx_, Feb 27, 2022.

  1. Nyx_

    Nyx_ New Member HowtoForge Supporter

    Hey Folks. Good day.
    I'm wondering if someone here can help me with a custom Fail2Ban Jail on my ISPConfig install.

    My goal is to block IPs that are trying to scan the webserver with several non-existing URLs. I can easily detect it by filtering by the 404 pages on Apache Logs.


    I created a filter on /etc/fail2ban/filter.d/apache-404.conf with this contents:
    Code:
    [Definition]
    failregex  =^<HOST>.*"(GET|POST|HEAD).*HTTP.*" 404 .*$
    ignoreregex =.*(robots.txt|favicon.ico|jpg|png)
    

    And added it's configuration on /etc/fail2ban/jail.local
    Code:
    [apache-404]
    # Custom Config - Ban IP on excessive 404 errors
    enabled = true
    port = http,https
    filter = apache-404
    logpath = %(apache_access_log)s
    bantime = 1d
    findtime = 1h
    maxretry = 3
    

    And, to test if it's working, I used the following:
    Code:
    # fail2ban-regex -v /var/log/apache2/other_vhosts_access.log /etc/fail2ban/filter.d/apache-404.conf --print-all-matched
    
    Running tests
    =============
    
    Use   failregex filter file : apache-404, basedir: /etc/fail2ban
    Use         log file : /var/log/apache2/other_vhosts_access.log
    Use         encoding : UTF-8
    
    
    Results
    =======
    
    Failregex: 2 total
    |-  #) [# of hits] regular expression
    |   1) [2] ^<HOST>.*"(GET|POST|HEAD).*HTTP.*" 404 .*$
    |      my.server.public.ip  Sun Feb 27 16:45:22 2022
    |      my.server.public.ip  Sun Feb 27 16:46:03 2022
    `-
    
    Ignoreregex: 55 total
    |-  #) [# of hits] regular expression
    |   1) [55] .*(robots.txt|favicon.ico|jpg|png)
    `-
    
    Date template hits:
    |- [# of hits] date format
    |  [6608] Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
    |  [0] {^LN-BEG}ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|  ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
    |  [0] {^LN-BEG}(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
    |  [0] {^LN-BEG}(?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
    |  [0] {^LN-BEG}Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2) %k:Minute:Second
    |  [0] {^LN-BEG}Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
    |  [0] {^LN-BEG}Month/Day/ExYear:24hour:Minute:Second
    |  [0] {^LN-BEG}Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
    |  [0] {^LN-BEG}Epoch
    |  [0] {^LN-BEG}ExYear2ExMonthExDay  ?24hour:Minute:Second
    |  [0] {^LN-BEG}MON Day, ExYear 12hour:Minute:Second AMPM
    |  [0] {^LN-BEG}ExYearExMonthExDay(?:T|  ?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
    |  [0] {^LN-BEG}(?:Zone name )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
    |  [0] {^LN-BEG}(?:Zone offset )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
    |  [0] {^LN-BEG}TAI64N
    |  [0] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|  ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
    |  [0] (?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
    |  [0] (?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
    |  [0] Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2) %k:Minute:Second
    |  [0] Month/Day/ExYear:24hour:Minute:Second
    |  [0] Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
    |  [0] Epoch
    |  [0] {^LN-BEG}24hour:Minute:Second
    |  [0] ^<Month/Day/[email protected]:Minute:Second>
    |  [0] ExYear2ExMonthExDay  ?24hour:Minute:Second
    |  [0] MON Day, ExYear 12hour:Minute:Second AMPM
    |  [0] ^MON-Day-ExYear2 %k:Minute:Second
    |  [0] ExYearExMonthExDay(?:T|  ?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
    |  [0] (?:Zone name )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
    |  [0] (?:Zone offset )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
    |  [0] TAI64N
    `-
    
    Lines: 6637 lines, 55 ignored, 2 matched, 6536 missed
    [processed in 7.00 sec]
    
    |- Matched line(s):
    |  myhosteddomain:443 client.public.ip - - [27/Feb/2022:16:45:22 -0300] "GET /3736363 HTTP/1.1" 404 6268 "-" "Mozilla/5.0 (Android 12; Mobile; rv:97.0) Gecko/97.0 Firefox/97.0"
    |  myhosteddomain:443 client.public.ip - - [27/Feb/2022:16:46:03 -0300] "GET /djdhdhd HTTP/1.1" 404 1101 "-" "Mozilla/5.0 (Android 12; Mobile; rv:97.0) Gecko/97.0 Firefox/97.0"
    `-
    


    Analyzing this output I can notice that, although the correct IP is logged on the Apache log, the IP identified by Fail2Ban is my server public IP (instead of the client/requestor public IP).

    Looking at the /var/log/fail2ban.log, I can confirm that the detected (and consequently the blocked) IP is not being identified correctly.

    Code:
    2022-02-27 16:45:23,227 fail2ban.ipdns          [122503]: WARNING Determined IP using DNS Lookup: myhosteddomain = {'my.server.public.ip'}
    2022-02-27 16:45:23,228 fail2ban.filter         [122503]: INFO    [apache-404] Found my.server.public.ip - 2022-02-27 16:45:03
    2022-02-27 16:46:04,327 fail2ban.ipdns          [122503]: WARNING Determined IP using DNS Lookup: myhosteddomain = {'my.server.public.ip'}
    2022-02-27 16:46:04,328 fail2ban.filter         [122503]: INFO    [apache-404] Found my.server.public.ip - 2022-02-27 16:46:03
    I'm wondering what's missing on this custom Jail to make it work correctly.
    I have other jails enabled (ssh, postfix, etc), all working fine.

    I appreciate if somebody can help with this.

    By the way, Apologies in advance. I know this is not 100% related to ISPConfig, if this is unaccepted on this forum, please, let me know. I don't mean to break any of the established rules.

    Thank you.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    I've moved your post to server operation forum.
     
  3. Nyx_

    Nyx_ New Member HowtoForge Supporter

    Thanks @till
    Folks, Any idea where/what I can look into to fix this issue?
    Thank you.
     
  4. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    You need to set prefregex (or fix your failregex); try starting with a working example like apache-auth.conf which includes apache-common.conf in order to reuse some existing regex definitions. (In a quick browse over filter.d/apache-*.conf I see other files which made the same mistake as you and I can't imagine would work as-is, so do be sure you start from a working example.)

    See https://github.com/fail2ban/fail2ban/wiki/Developing-Regex-in-Fail2ban
     
    Last edited: Mar 2, 2022

Share This Page