Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > Linux Forums > Programming/Scripts

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 10th January 2012, 15:59
baldur2630 baldur2630 is offline
Member
 
Join Date: Jan 2007
Location: Belgium
Posts: 30
Thanks: 3
Thanked 1 Time in 1 Post
Unhappy Urgent help needed with failregex expression

I've been using fail2ban for a long time on CentOS 5 and it's worked like a charm.

I recently installed a new CentOS 6.2 Server and moved my websites and forums onto that, now life has become a nightmare because we are being bombarded 24 x 7 by moronic scriptkiddies. It's so bad the entire system went down over the Christmas period and my fail2ban expressions don't work any longer. I'm not a programmer, but I see that the format of the entries in the log files are different!

I'm getting different errors in the error logs : -

[Mon Jan 09 14:47:27 2012] [error] [client 173.212.213.56] File does not exist: /var/www/xxmusic/components/com_galleria
[Mon Jan 09 14:54:49 2012] [error] [client 212.13.239.86] File does not exist: /var/www/xxmusic/muieblackcat

and

[Tue Jan 10 13:49:16 2012] [error] [client 96.127.137.26] script '/var/www/xxmusic/site.php' not found or unable to stat
[Tue Jan 10 13:49:17 2012] [error] [client 96.127.137.26] script '/var/www/xxmusic/site.php' not found or unable to stat

On the old server, fail2ban caught all of these, on the new server ZERO and we are getting thousands of these 24 x 7

I used a filter.d called apache-noscript on the old server and another called apache-nohome.

My apache-noscript expression was : failregex = [[]client <HOST>[]] (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)

and the apache-nohome was : failregex = [[]client <HOST>[]] File does not exist: .*/~.*


Can someone PLEASE help me to get 2 x failregex expressions that will work?
Reply With Quote
Sponsored Links
  #2  
Old 11th January 2012, 22:21
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Please post here:

From the working server:
1) a few log lines
2) working regex

From the not working server:
1) a few log lines
2) not working regex

In everycase, you can use fail2ban-regex command to test your regex and try to get a working one.

fail2ban-regex /path/to/logfile "regex to be evaluted by fail2ban"

It will show matches.. What I do, is just to paste a line in a file /test/test.log, and then run the check
Reply With Quote
  #3  
Old 12th January 2012, 14:40
baldur2630 baldur2630 is offline
Member
 
Join Date: Jan 2007
Location: Belgium
Posts: 30
Thanks: 3
Thanked 1 Time in 1 Post
Default

Sorry, I was away yesterday.

The server that worked OK was trashed, so I don't have the information you asked for.

These are the kind of attacks we are getting : -

[Sat[Sat Jan 07 19:49:46 2012] [error] [client 173.212.195.166] File does not exist: /var/www/hktmusic/components/com_madeira
[Sat Jan 07 20:42:18 2012] [error] [client 173.212.209.238] File does not exist: /var/www/hktmusic/components/com_moodle
[Sat Jan 07 20:50:15 2012] [error] [client 173.212.197.252] File does not exist: /var/www/hktmusic/administrator/components/ Jan 07 18:23:04 2012] [error] [client 197.109.34.193] PHP Notice: Trying to get property of non-object in /var/www/hktmusic/components/com_mymuse/helpers/checkout.php on line 698

[Mon Jan 09 09:02:16 2012] [error] [client 173.212.209.238] script '/var/www/hktmusic/modules/mod_calendar.php' not found or unable to stat
[Sun Jan 08 23:29:19 2012] [error] [client 192.168.0.23] script '/var/www/techsup/ntforum/htpath.php' not found or unable to stat
[Mon Jan 09 01:23:29 2012] [error] [client 184.173.185.234] File does not exist: /var/www/techsup/ntforum/+[PLM=0][N]+GET+http:, referer: http://techsup.corp.networkingtechno...3E+%5BN%5D+GET
+http://techsup.corp.networkingtechno...22450,0,361%5D

The fail2ban in this case seems to work, but it doesn't ban anything!

Test gives me : -

[root@centos-62 ~]# fail2ban-regex /var/log/httpd/hktmusic-error_log /etc/fail2ban/filter.d/apache-pma.conf
/usr/share/fail2ban/server/filter.py:442: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import md5

Running tests
=============

Use regex file : /etc/fail2ban/filter.d/apache-pma.conf
Use log file : /var/log/httpd/hktmusic-error_log


Results
=======

Failregex
|- Regular expressions:
| [1] [[]client <HOST>[]] (File does not exist|script not found or unable to stat): .*/(cgi-bin|admin|Admin|sql|mail|phpmyadmin|file:|php|pma| web|PMA|PMA2006|pma2006|sqlmanager|mysqlmanager|PM A2005|phpmyadmin-old|phpmyadminold|pma2005|phpmanager|mysql|myadmin |webadmin|sqlweb|websql|webdb|mysqladmin|mysql-admin|phpmyadmin2|phpMyAdmin2|phpMyAdmin-2|php-my-admin|cms|clan|site|seite|page|forum|wbb2|board|wb b|archive|forumv2|forumv1|b0ard|f0rum|wbb1|wbb3|wb blite|directforum|board23|board2|board3|WBB|WBB2|h tml|phpkit|page|phpkit_1.6.1|clan|myadmin|webadmin |sqlweb|websql|webdb|mysqladmin|mysql-admin|phpmyadmin2|php-my-admin|phpMyAdmin-2.2.3|phpMyAdmin-2.2.6|phpMyAdmin-2.5.1|phpMyAdmin-2.5.4|phpMyAdmin-2.5.6|phpMyAdmin-2.6.0|phpMyAdmin-2.6.0-pl1|phpMyAdmin-2.6.2-rc1|phpMyAdmin-2.6.3|phpMyAdmin-2.6.3-pl1|phpMyAdmin-2.6.3-rc1|padmin|datenbank|ZenCart|cart|commerce|e-commerce|shop|stories|store|zc|dbadmin|typo3|datab ase|horde|horde2|horde3|horde-3.0.9|Horde|README|horde-3.0.9|adserver|phpAdsNew|phpadsnew|phpads|Ads|ads| xmlrpc|xmlsrv|blog|drupal|community|blogs|blogtest |appserver|roundcube|rc|mail|mail2|roundcubemail|r ms|webmail2|webmail|wm|bin|roundcubemail-0.1|roundcubemail-0.2|roundcube-0.1|roundcube-0.2|roun|cube|wp-login.php|ucp.php|\.asp|\.dll|\.exe|\.pl)
|
`- Number of matches:
[1] 95 match(es)

Ignoreregex
|- Regular expressions:
|
`- Number of matches:

Summary
=======

Addresses found:
[1]
173.212.195.22 (Sun Jan 08 04:34:34 2012)
64.191.99.103 (Sun Jan 08 04:56:39 2012)
173.212.209.202 (Sun Jan 08 05:02:04 2012)
64.191.99.70 (Sun Jan 08 05:07:32 2012)
173.212.209.238 (Sun Jan 08 09:41:41 2012)
66.197.166.86 (Sun Jan 08 09:46:11 2012)
173.212.209.220 (Sun Jan 08 10:13:33 2012)
173.212.195.170 (Sun Jan 08 11:54:57 2012)
64.191.99.103 (Sun Jan 08 14:25:44 2012)
173.212.195.176 (Sun Jan 08 14:48:04 2012)
173.212.209.220 (Sun Jan 08 16:43:49 2012)
173.212.209.202 (Sun Jan 08 16:51:09 2012)
64.191.99.103 (Sun Jan 08 17:09:33 2012)
96.9.173.32 (Mon Jan 09 02:01:50 2012)
173.212.209.202 (Mon Jan 09 02:32:54 2012)
173.212.209.220 (Mon Jan 09 03:05:54 2012)
173.212.209.212 (Mon Jan 09 03:14:08 2012)
173.212.209.212 (Mon Jan 09 04:27:08 2012)
96.9.173.4 (Mon Jan 09 05:05:06 2012)
173.212.209.220 (Mon Jan 09 06:04:28 2012)
173.212.209.212 (Mon Jan 09 07:00:57 2012)
173.212.209.220 (Mon Jan 09 07:31:32 2012)
173.212.209.212 (Mon Jan 09 08:35:18 2012)
96.9.173.32 (Mon Jan 09 10:34:09 2012)
173.212.213.56 (Mon Jan 09 13:58:53 2012)
212.13.239.86 (Mon Jan 09 14:54:51 2012)
212.13.239.86 (Mon Jan 09 14:54:51 2012)
212.13.239.86 (Mon Jan 09 14:54:52 2012)
212.13.239.86 (Mon Jan 09 14:54:53 2012)
212.13.239.86 (Mon Jan 09 14:54:53 2012)
212.13.239.86 (Mon Jan 09 14:54:53 2012)
212.13.239.86 (Mon Jan 09 14:54:54 2012)
212.13.239.86 (Mon Jan 09 14:54:54 2012)
212.13.239.86 (Mon Jan 09 14:54:55 2012)
212.13.239.86 (Mon Jan 09 14:55:01 2012)
212.13.239.86 (Mon Jan 09 14:55:01 2012)
212.13.239.86 (Mon Jan 09 14:55:02 2012)
212.13.239.86 (Mon Jan 09 14:55:02 2012)
212.13.239.86 (Mon Jan 09 14:55:03 2012)
212.13.239.86 (Mon Jan 09 14:55:03 2012)
212.13.239.86 (Mon Jan 09 14:55:04 2012)
212.13.239.86 (Mon Jan 09 14:55:04 2012)
212.13.239.86 (Mon Jan 09 14:55:05 2012)
212.13.239.86 (Mon Jan 09 14:55:05 2012)
212.13.239.86 (Mon Jan 09 14:55:06 2012)
212.13.239.86 (Mon Jan 09 14:55:06 2012)
212.13.239.86 (Mon Jan 09 14:55:08 2012)
212.13.239.86 (Mon Jan 09 14:55:09 2012)
212.13.239.86 (Mon Jan 09 14:55:09 2012)
212.13.239.86 (Mon Jan 09 14:55:10 2012)
212.13.239.86 (Mon Jan 09 14:55:10 2012)
212.13.239.86 (Mon Jan 09 14:55:10 2012)
212.13.239.86 (Mon Jan 09 14:55:11 2012)
212.13.239.86 (Mon Jan 09 14:55:20 2012)
212.13.239.86 (Mon Jan 09 14:55:21 2012)
173.212.213.56 (Mon Jan 09 15:34:09 2012)
173.212.195.166 (Mon Jan 09 15:59:22 2012)
64.191.99.107 (Mon Jan 09 16:14:06 2012)
96.9.173.32 (Mon Jan 09 17:06:15 2012)
173.212.209.212 (Mon Jan 09 19:17:52 2012)
173.212.209.202 (Tue Jan 10 03:16:13 2012)
64.191.99.103 (Tue Jan 10 03:23:22 2012)
96.9.173.32 (Tue Jan 10 03:47:15 2012)
173.212.195.162 (Tue Jan 10 08:31:20 2012)
173.212.195.162 (Tue Jan 10 09:06:08 2012)
96.9.173.32 (Tue Jan 10 09:10:15 2012)
96.9.173.32 (Tue Jan 10 12:24:28 2012)
96.9.173.32 (Tue Jan 10 16:28:29 2012)
96.9.173.4 (Tue Jan 10 17:39:20 2012)

Date template hits:
314 hit(s): MONTH Day Hour:Minute:Second
0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second Year
0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second
0 hit(s): Year/Month/Day Hour:Minute:Second
0 hit(s): Day/Month/Year Hour:Minute:Second
0 hit(s): Day/MONTH/Year:Hour:Minute:Second
0 hit(s): Month/Day/Year:Hour:Minute:Second
0 hit(s): Year-Month-Day Hour:Minute:Second
0 hit(s): Day-MONTH-Year Hour:Minute:Second[.Millisecond]
0 hit(s): Day-Month-Year Hour:Minute:Second
0 hit(s): TAI64N
0 hit(s): Epoch
0 hit(s): ISO 8601
0 hit(s): Hour:Minute:Second
0 hit(s): <Month/Day/Year@Hour:Minute:Second>

Success, the total number of match is 95

However, look at the above section 'Running tests' which could contain important information.

This is the entry in filter.d : -

# Fail2Ban configuration file
#
# Author: Remco Overdijk
#
# $Revision: 4 $
#

[Definition]

# Option: failregex
# Notes.: regex to match the 404'ed PMA file in the logfile. The
# host must be matched by a group named "host". The tag "<HOST>" can
# be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}?(?P<host>\S+)
# Values: TEXT
#
failregex = [[]client <HOST>[]] (File does not exist|script not found or unable to stat): .*/(cgi-bin|admin|Admin|sql|mail|phpmyadmin|file:|php|pma| web|PMA|PMA2006|pma2006|sqlmanager|mysqlmanager|PM A2005|phpmyadmin-old|phpmyadminold|pma2005|phpmanager|mysql|myadmin |webadmin|sqlweb|websql|webdb|mysqladmin|mysql-admin|phpmyadmin2|phpMyAdmin2|phpMyAdmin-2|php-my-admin|cms|clan|site|seite|page|forum|wbb2|board|wb b|archive|forumv2|forumv1|b0ard|f0rum|wbb1|wbb3|wb blite|directforum|board23|board2|board3|WBB|WBB2|h tml|phpkit|page|phpkit_1.6.1|clan|myadmin|webadmin |sqlweb|websql|webdb|mysqladmin|mysql-admin|phpmyadmin2|php-my-admin|phpMyAdmin-2.2.3|phpMyAdmin-2.2.6|phpMyAdmin-2.5.1|phpMyAdmin-2.5.4|phpMyAdmin-2.5.6|phpMyAdmin-2.6.0|phpMyAdmin-2.6.0-pl1|phpMyAdmin-2.6.2-rc1|phpMyAdmin-2.6.3|phpMyAdmin-2.6.3-pl1|phpMyAdmin-2.6.3-rc1|padmin|datenbank|ZenCart|cart|commerce|e-commerce|shop|stories|store|zc|dbadmin|typo3|datab ase|horde|horde2|horde3|horde-3.0.9|Horde|README|horde-3.0.9|adserver|phpAdsNew|phpadsnew|phpads|Ads|ads| xmlrpc|xmlsrv|blog|drupal|community|blogs|blogtest |appserver|roundcube|rc|mail|mail2|roundcubemail|r ms|webmail2|webmail|wm|bin|roundcubemail-0.1|roundcubemail-0.2|roundcube-0.1|roundcube-0.2|roun|cube|wp-login.php|ucp.php|\.asp|\.dll|\.exe|\.pl)
# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =

This is the entry for the above filter in jail.conf :-

[apache-pma]
enabled = true
filter = apache-pma
action = iptables-allports[name=pma]
sendmail-whois[name=php-attack, dest=hmartin@networkingtechnology.org]
logpath = /var/log/httpd/techsup-error_log
logpath = /var/log/httpd/mlamusic-error_log
logpath = /var/log/httpd/hktmusic-error_log
maxretry = 1

The ban time etc., is set to : -
# "bantime" is the number of seconds that a host is banned.
bantime = 31536000

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime = 600

# "maxretry" is the number of failures before a host get banned.
maxretry = 3

i've also got several other filters which I've tried and they don't work either. The attacks pour in but fail3ban just doesn't work any longer.

I tried apache-noscript.conf - this kills fail2ban : -


failregex = [[]client <HOST>[]] (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)
[[]client <HOST>[]] script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat *$

I tried apache-nohome.conf

# failregex = [[]client <HOST>[]] File does not exist:
# failregex = [[]client (?P<host>\S*)[]] File does not exist:
# failregex = [[]client <HOST>[]] File does not exist: .*/~.*
# failregex = [[]client ?P<host>[]] File does not exist: .*\.php

this also kills fail2ban

I tried apache-404.conf : -

failregex = (?P<HOST>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) .+ 404 [0-9]+ "

also kills fail2ban.

I've scoured the web and tried every version I could find which might work. the ONLY one that gives me anything with testing is apache-pma, but it doesn't ban anything at all.
Reply With Quote
  #4  
Old 12th January 2012, 17:20
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Could you post you iptables --list pls
Reply With Quote
  #5  
Old 12th January 2012, 18:47
baldur2630 baldur2630 is offline
Member
 
Join Date: Jan 2007
Location: Belgium
Posts: 30
Thanks: 3
Thanked 1 Time in 1 Post
Default

[root@centos-62 ~]# iptables --list
Chain INPUT (policy ACCEPT)
target prot opt source destination
fail2ban-sasl tcp -- anywhere anywhere tcp dpt:smtp
fail2ban-pma tcp -- anywhere anywhere
fail2ban-ProFTPD tcp -- anywhere anywhere tcp dpt:ftp
fail2ban-webmin tcp -- anywhere anywhere tcp dpt:ndmp
fail2ban-BadBots tcp -- anywhere anywhere multiport dports http,https
fail2ban-SSH tcp -- anywhere anywhere tcp dpt:ssh
fail2ban-PHP-fopen tcp -- anywhere anywhere multiport dports http,https
fail2ban-default tcp -- anywhere anywhere
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp
ACCEPT udp -- anywhere anywhere state NEW udp dpt:netbios-ns
ACCEPT udp -- anywhere anywhere state NEW udp dpt:netbios-dgm
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:netbios-ssn
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:microsoft-ds
ACCEPT udp -- anywhere anywhere state NEW udp dpt:netbios-ns
ACCEPT udp -- anywhere anywhere state NEW udp dpt:netbios-dgm
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp-data
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:mysql
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain fail2ban-BadBots (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-PHP-fopen (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-ProFTPD (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-SSH (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-default (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-pma (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-sasl (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-webmin (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Reply With Quote
  #6  
Old 12th January 2012, 19:41
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Please, just to try this:

1) Reduce "bantime" to 600 seconds.
2) Comment out with a # the line "action" in jail.conf, and add a line "port = http,https"

With iptables --list you must see after fail2ban restart as following:

fail2ban-pma tcp -- anywhere anywhere multiport dports http,https

instead of:

fail2ban-pma tcp -- anywhere anywhere

3) You have a duplicated "maxretry", delete one.
4) restart fail2ban, and try to access a few times the website using this line:

domain.com/phpmanager

You should get banned (because regex is working perfectly as you tested).

Logged in by ssh, when you get banned, use iptables --list again, and your ip must be listed in the following chain:

Chain fail2ban-pma (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Is not working and you are not banned, check the log file that fail2ban is using, and see if your attempt to enter to /phpmanager was logged correctly, and post line here.

To be debanned, you just have to restart fail2ban.

Post results pls.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
dovecot jacksmerv Installation/Configuration 6 24th December 2010 11:53
Error installing Dovecot dbetts22 Installation/Configuration 6 19th August 2010 18:07
Failed Dependencies albertux Installation/Configuration 9 17th August 2010 22:14
Upgrade to php 5.2 in suse 10.1 gimhan90 Installation/Configuration 3 1st January 2009 20:32
odbc Chad Installation/Configuration 0 10th April 2008 01:43


All times are GMT +2. The time now is 01:52.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.