Firewall/DNS issue?

[cly]

I have installed ISPConfig3 and used it for a couple of months successfully on 2 seperate VPS servers at the same ISP - one as a web/mysql/ftp/dns server, and the other as a mail/dns server. Just recently, for some reason, the web server has, what I believe, a firewall issue, but I can't figure out what is going on:

Configuration:
Server A (with the problem): Web/mysql/ftp/dns
Firewall configuration: TCP 20,21,22,25,53,80,443,3306,8080,8081 UDP 53,161,3306
Debian 6 64bit on OpenVZ

Server B (works fine): mail/dns
Firewall configuration: TCP 22,25,53,110,143,3306 UDP 53,161,3306
Debian 6 64bit on OpenVZ

What happens:
1) Server A has login delays of 15 seconds between entering username and password
2) Cannot ping/resolve any name from Server A (no name resolution)
3) Can ping IP addresses fine
4) If I telnet to a DNS server on port 53, it fails unless the firewall is disabled, even though both TCP and UDP 53 are configured on the firewall.
4) If the firewall is disabled, everything works fine - name resolution and fast logins

The first time I built Server A it worked fine the whole time. I installed SNMPD and it stopped working, so thought it might have been that, but it appears that it may have been a coincidence. So I rebuilt the server, and as soon as the firewall is turned on, the problem comes back. There is no such problem with Server B. I have deleted the firewall rules and recreated them (and even rebuilt the whole server).

Both servers have the same resolv.conf, and Server A works fine with the firewall disabled and Sever B works fine all the time.

Any help would be appreciated

[till]

Maybe there is some kind of iptables conflict between the ispconfig firewall and another softeare. The bastille firewall which is used in ispconfig does not block any outgoing connections.

Please stop fail2ban and then start the firewall again and check if it works. If it does not work, please post the output of:

iptables -L

when the ispconfig firewall is turned on.

[cly]

Originally Posted by till

Please stop fail2ban and then start the firewall again and check if it works. If it does not work, please post the output of:

iptables -L

when the ispconfig firewall is turned on.

I stopped fail2ban and confirmed that it wasn't going, and the problem does still exist unfortunately.

Below is the output from iptables as requested:

Chain INPUT (policy DROP)
target prot opt source destination
DROP tcp -- anywhere loopback/8
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
DROP all -- 224.0.0.0/4 anywhere
PUB_IN all -- anywhere anywhere
PUB_IN all -- anywhere anywhere
PUB_IN all -- anywhere anywhere
PUB_IN all -- anywhere anywhere
DROP all -- anywhere anywhere

Chain FORWARD (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
DROP all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
PUB_OUT all -- anywhere anywhere
PUB_OUT all -- anywhere anywhere
PUB_OUT all -- anywhere anywhere
PUB_OUT all -- anywhere anywhere

Chain INT_IN (0 references)
target prot opt source destination
ACCEPT icmp -- anywhere anywhere
DROP all -- anywhere anywhere

Chain INT_OUT (0 references)
target prot opt source destination
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere

Chain PAROLE (11 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere

Chain PUB_IN (4 references)
target prot opt source destination
ACCEPT icmp -- anywhere anywhere icmp destination-unreachable
ACCEPT icmp -- anywhere anywhere icmp echo-reply
ACCEPT icmp -- anywhere anywhere icmp time-exceeded
ACCEPT icmp -- anywhere anywhere icmp echo-request
PAROLE tcp -- anywhere anywhere tcp dpt:ftp-data
PAROLE tcp -- anywhere anywhere tcp dpt:ftp
PAROLE tcp -- anywhere anywhere tcp dpt:ssh
PAROLE tcp -- anywhere anywhere tcp dpt:smtp
PAROLE tcp -- anywhere anywhere tcp dpt:domain
PAROLE tcp -- anywhere anywhere tcp dpt:www
PAROLE tcp -- anywhere anywhere tcp dpt:https
PAROLE tcp -- anywhere anywhere tcp dpt:mysql
PAROLE tcp -- anywhere anywhere tcp dpt:http-alt
PAROLE tcp -- anywhere anywhere tcp dpt:tproxy
PAROLE tcp -- anywhere anywhere tcp dpts:41000:41100
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:snmp
ACCEPT udp -- anywhere anywhere udp dpt:mysql
DROP icmp -- anywhere anywhere
DROP all -- anywhere anywhere

Chain PUB_OUT (4 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere

There's also one other rule in there (41000:41100) that I noticed I had omitted - that is for passive FTP.

[falko]

Can you disable the firewall completely for testing purposes so that we can see if it really is the problem?

[cly]

Sure, below are some quick tests with the firewall ENABLED:


iptables -L
(as above, to confirm enabled)


root@hydrogen:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=51 time=1.77 ms
64 bytes from 8.8.8.8: icmp_req=2 ttl=51 time=1.94 ms
64 bytes from 8.8.8.8: icmp_req=3 ttl=51 time=10.3 ms

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.776/4.683/10.328/3.992 ms

root@hydrogen:~# ping google.com
ping: unknown host google.com

root@hydrogen:~# nslookup google.com
;; connection timed out; no servers could be reached


And then the same tests again with the DISABLED firewall:


iptables -L (With fail2ban enabled)
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain fail2ban-pureftpd (0 references)
target prot opt source destination
RETURN all -- anywhere anywhere

Chain fail2ban-ssh (0 references)
target prot opt source destination
RETURN all -- anywhere anywhere

root@hydrogen:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=51 time=1.67 ms
64 bytes from 8.8.8.8: icmp_req=2 ttl=51 time=1.69 ms
64 bytes from 8.8.8.8: icmp_req=3 ttl=51 time=2.77 ms

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 1.678/2.050/2.773/0.511 ms

root@hydrogen:~# ping google.com
PING google.com (74.125.237.49) 56(84) bytes of data.
64 bytes from 74.125.237.49: icmp_req=1 ttl=50 time=1.78 ms
64 bytes from 74.125.237.49: icmp_req=2 ttl=50 time=3.24 ms
64 bytes from 74.125.237.49: icmp_req=3 ttl=51 time=69.2 ms

--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.788/24.759/69.243/31.460 ms

root@hydrogen:~# nslookup google.com
Server: x.x.x.x
Address: x.x.x.x#53 (ISP's DNS server, as per resolv.conf)


Non-authoritative answer:
Name: google.com
Address: 74.125.237.48
Name: google.com
Address: 74.125.237.49
Name: google.com
Address: 74.125.237.50
Name: google.com
Address: 74.125.237.51
Name: google.com
Address: 74.125.237.52


As you will see, a simple disable of the firewall gets name resolution working, but to me iptables appears to be working fine (however, perhaps I am missing something obvious).

Any help would be appreciated as I am scratching my head here

[cly]

As an update, I rebuilt the server without fail2ban, and the same problem exists, so it is not fail2ban causing issues. I also have rebuilt the server to 32bit to see if it was anything to do with my ISP's 64bit Debian 6 OpenVZ template, but the same thing still happens

[falko]

What's the output of

cat /proc/user_beancounters

?

[cly]

Originally Posted by falko

What's the output of cat /proc/user_beancounters ?

Version: 2.5
uid resource held maxheld barrier limit failcnt
3082: kmemsize 12102839 34295580 2147483646 2147483646 0
lockedpages 0 1002 999999 999999 0
privvmpages 114509 263212 262144 262144 14648
shmpages 1078 9717 131072 131072 0
dummy 0 0 0 0 0
numproc 56 106 999999 999999 0
physpages 34555 129442 0 2147483647 0
vmguarpages 0 0 131072 2147483647 0
oomguarpages 34555 129442 131072 2147483647 0
numtcpsock 13 61 7999992 7999992 0
numflock 6 27 999999 999999 0
numpty 1 7 500000 500000 0
numsiginfo 1 30 999999 999999 0
tcpsndbuf 227552 1085072 214748160 396774400 0
tcprcvbuf 212992 1991032 214748160 396774400 0
othersockbuf 32824 373040 214748160 396774400 0
dgramrcvbuf 0 17504 214748160 396774400 0
numothersock 24 157 7999992 7999992 0
dcachesize 441330 850734 2147483646 2147483646 0
numfile 1424 3368 23999976 23999976 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
numiptent 69 78 999999 999999 0

[falko]

privvmpages 114509 263212 262144 262144 14648

Looks like you should assign more memory to your VPS.

[cly]

Originally Posted by falko

Looks like you should assign more memory to your VPS.

I understand that it looks that way, but there doesn't actually appear to be a RAM issue as far as I can see, unless I am looking at it incorrectly. If I run top (see results below), there is still 578MB out of 1GB free. The failcnt figure has been static for a while, even after enabling/disabling the firewall, and performing tests. Even after about a day since I posted the failcnt figure, it is still exactly the same at 14648.

top - 09:02:25 up 12:19, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 34 total, 1 running, 33 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1048576k total, 456760k used, 591816k free, 0k buffers
Swap: 0k total, 0k used, 0k free, 0k cached

Also, why would a simple enable/disable of the firewall change the RAM usage so much as to stop only DNS resolution? If I use my hosts file and browse directly to a website I have hosted on the server, it works absolutely fine, so it only appears to be DNS that is not working through the firewall.

Sorry - I am confused about this.