Squid Proxy Server On Ubuntu 9.04 Server With DansGuardian, ClamAV, And WPAD (Proxy Auto-Detection) [Updated 07-08-09]

Want to support HowtoForge? Become a subscriber!
 
Submitted by mtcoombes (Contact Author) (Forums) on Wed, 2009-07-01 18:25. :: Ubuntu | Security

Squid Proxy Server on Ubuntu 9.04 with DansGuardian, ClamAV, and WPAD (Proxy Auto-Detection)

Version 1.1
Mark Coombes (mtcoombes at gmail dot com)

Changelog:
1.0 - First version of this tutorial
1.1 - Added information on achieving WPAD through dhcp3; added section on lighttpd; added information on SSH

This tutorial will demonstrate how to set up a Squid Proxy server on Ubuntu 9.04 with DansGuardian (for content filtering) and ClamAV (for Virus scanning); in addition, we will set up Web Proxy AutoDetection (WPAD) through DHCP (in this case, either dhcp3 OR the Windows Server 2003 DHCP server) or DNS so that the only configuration necessary on the client side is to check "Auto-detect proxy settings for this network" in Firefox or Internet Explorer. At the end of the tutorial, users will have a fully functional and secure proxy for HTTP access.

In addition... the final section will show users how to connect to this proxy server via SSH in order to have a protected browsing session while away from the home/office.

NOTE: This tutorial does NOT create a transparent proxy! Internet traffic other than that passed through a web browser will not be proxied! These are the steps that worked for me, but I make no claim that they will work for you without modification.

Preliminary Configuration

This tutorial assumes that you have already installed Ubuntu Server 9.04, updated and upgraded installed packages to their latest versions. Preconfiguration of Squid, DansGuardian, ClamAV, dhcp3, and Apache or Lighttpd is not necessary - the following steps will show you all the configuration that is necessary. All installation and configuration must be done as a user with Sudo privileges.

1. Installing and Configuring ClamAV

Execute the following from a shell to install ClamAV:

sudo apt-get install clamav-daemon clamav-freshclam

This will install the ClamAV daemon necessary for virus scanning with DansGuardian, and also FreshClam to ensure that the virus database is always kept up-to-date. You may get the following warning:

LibClamAV Warning:***********************************************************
LibClamAV Warning: *** This version of the ClamAV engine is outdated.***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***
LibClamAV Warning: ***********************************************************

Which indicates that the virus engine itself is out-of-date because of the version packaged with Ubuntu 9.04. For the highest security, you should endeavour to keep the ClamAV engine updated to the latest version, but this is not possible through apt and so is out of the scope of this guide. For the most part, I believe that this warning can be ignored.

The configuration files for ClamAV are located in /etc/clamav, but the default configuration is appropriate for our needs. FreshClam will update virus definitions hourly; if you want to change this behaviour, edit /etc/clamav/freshclam.conf and change the section:

# Check for new database 24 times a day
Checks 24

To the number of times you want to check for updates, e.g.:

# Check for new database 24 times a day
Checks 48

To do it twice as often. Press Ctrl+X to exit and answer Y when nano asks if you want to "Save the Modified Buffer." If you have changed the configuration of FreshClam, execute the following to ensure the new configuration is loaded:

sudo /etc/init.d/clamav-freshclam restart

2. Installing and Configuring Squid

Execute the following from a shell to install Squid:

sudo apt-get install squid

The configuration directives for Squid are located in /etc/squid/squid.conf. Edit this file now by typing:

sudo nano /etc/squid/squid.conf

We do not have to do much to the default configuration of Squid because DansGuardian will sit in-between Squid and the users and direct traffic to the appropriate Squid port. Set this port now by adding this parameter to the bottom of the file:

http_port 3128

Press Ctrl+X to exit and answer Y when nano asks if you want to "Save the Modified Buffer." Then type the following to have Squid reload its configuration file:

sudo /etc/init.d/squid reload

3. Installing and Configuring DansGuardian

Execute the following from a shell to install DansGuardian:

sudo apt-get install dansguardian

The configuration files for DansGuardian are located in /etc/dansguardian. Edit the main configuration now by typing:

sudo nano /etc/dansguardian/dansguardian.conf

The entire dansguardian.conf is well commented and informative. The following will demonstrate a basic set-up, but to really get DansGuardian's content-filtering abilities working for you, you should read this file thoroughly. By default, DansGuardian is very restrictive (which is likely what you are after); relaxing these restrictions involves some patience and editing of configuration files. I will touch on some of this below. Take note of the following lines in the configuration:

language = 'ukenglish'
filterip = x.x.x.x
filterport = 8080
proxyip = 127.0.0.1
proxyport = 3128
#contentscanner = '/etc/dansguardian/contentscanners/clamav.conf'

The "language" directive indicates the language of the error page DansGuardian will display when it blocks a web page. Possible values for this option are found in the /etc/dansguardian directory. Once you have decided upon the language of the error page you would like to display to your users, change the language string in /etc/dansguardian/dansguardian.conf; and afterwards customize the error page:

sudo nano /etc/dansguardian/languages/ukenglish/template.html

Follow the instructions in template.html to insert your company name, and to customize the message for your purposes.

The "filterip" and "filterport" directives define, respectively, the ip address and port of the proxy server. Set filterip to equal the ip that this machine has on your LAN, and filterport equal to the port you want the proxy to have; a good default is 8080.

Important: If you want to be able to access this proxy remotely via an SSH server on the same machine, add another "filterip" line, and use the IP 127.0.0.1 - this will ensure that DansGuardian will also listen to localhost also for connection requests.

The "proxyip" directive is the IP address of the Squid proxy and "proxyport" is the port that the proxy listens on. If you have been following this guide, leave it at 127.0.0.1, in other words, the local machine. Similarly, if you used the default Squid port suggested earlier, you can leave proxyport as 3128.

The "contentscanner" line is already in /etc/dansguardian/dansguardian.conf towards the bottom. Uncomment this line to tell DansGuardian to use ClamAV to scan items requested via HTTP.

Finally, comment out or delete the line:

UNCONFIGURED - Please remove this line after configuration

To tell DansGuardian that you have changed the default configuration. Save this file using Ctrl+X and "Y", then do:

sudo /etc/init.d/dansguardian restart

To have DansGuardian reread its configuration and connect to the running Squid proxy.

At this point, do the following to secure the Squid proxy and prevent users from circumventing DansGuardian:
NOTE: Do NOT do this if you have other firewall rules in place! This will mess them up. Simply adapt whatever other firewall method you are using to take into account the following.

sudo ufw default DENY
sudo ufw ALLOW 8080
sudo ufw enable

This ensures that all incoming connections other than to port 8080 (the DansGuardian proxy port) are blocked (preventing sneaky users from accessing the open Squid proxy on port 3128). Alternately, you could explicitly block 3128 with:

sudo ufw DENY 3128

But I chose the first way to make the system more secure overall.

The directory /etc/dansguardian/lists contains all of the files related to content-filtering. Read through each of these files carefully to get an idea of the shaping possible with DansGuardian, and also to relax or tighten the strictness of DansGuardian's defaults. Since we are using ClamAV in conjunction with DansGuardian, a directory worth checking out is /etc/dansguardian/lists/contentscanners, which contains exception lists for the files that you don't want to be checked by ClamAV; several exceptions are listed in these files by default, so they are worth checking to ensure your needs are met!

4. Setting up Web Proxy Auto Detection (WPAD)

At this point, if everything has succeeded, you should be able to go into the proxy settings of your web browser and input the address and port of the proxy server we have just set up, and have a proxied connection protected by DansGuardian and ClamAV. We want this information to be provided automatically for users, so we must proceed with the following steps to set up Auto Detection; note that this procedure still requires you to set up the web browser initially (i.e., tick the "Automatically detect settings" box). For a more 'transparent' way to set up a proxy, you will need to look into Transparent proxying with Squid, but this is out of the scope of the current tutorial.

Essentially, WPAD works by serving a script to the browsers on your network via an http server. Since I want this machine to be an all-in-one solution, I will go ahead with installing a web server now (no fancy configuration is necessary, since users will not be interacting with the web server directly). If you have a web server already existing on your network, and clients can connect to it locally, you can also adapt this configuration to place the WPAD script on that server instead.

Web Server Option 1 - Apache2

First, we install Apache:

sudo apt-get install apache2

We need to add one line to /etc/apache2/httpd.conf; do so by executing:

sudo nano /etc/apache2/httpd.conf

And add the following line to the file (this associates the correct mime type for the autoconfiguration script to the .dat extension):

AddType application/x-ns-proxy-autoconfig .dat

Exit and save with Ctrl+X and "Y", then reload (or if Apache hasn't started, start it) with:

sudo /etc/init.d/apache2 reload
OR
sudo /etc/init.d/apache2 start

Now in a web browser navigate to the IP address of your server; a default Apache configuration should return "It Works!" which will let you know that the server is working!

Web Server Option 2 - lighttpd

The ideal web server for this application (if you won't be needing it for anything else) would appear to be lighttpd. I haven't tested lighttpd extensively, but the following section will help you to install and configure it to serve the WPAD script to clients. First, install lighttpd:

sudo apt-get install lighttpd

We need to associate the proper mime-type to the configuration script. To do so we must edit /etc/lighttpd/lighttpd.conf:

sudo nano /etc/lighttpd/lighttpd.conf

Add the following section to the bottom of the file (a default install should not have an existing mime-type section):

mimetype.assign = ( ".dat" => "application/x-ns-proxy-autoconfig" )

Ctrl-X and "Y" to exit nano and save the changes, then do:

sudo /etc/init.d/lighttpd restart

To restart lighttpd. Now in a web browser navigate to the IP address of your server and you should be greeted with the lighttpd default page, letting you know that the server is working!

Creating the Autoconfiguration Script "wpad.dat"

By default, the root of the web server (for either case) is in /var/www. Do the following to create the autoconfiguration script:

sudo touch /var/www/wpad.dat
sudo nano /var/www/wpad.dat

And paste the following into the file:

function FindProxyForURL(url, host)
{
if (isInNet(host, "x.x.x.0", "255.255.255.0"))
return "DIRECT";
else
return "PROXY x.x.x.x:8080";
}

There are many ways to create such a script, to achieve different purposes. The above script will need to be edited to match your configuration. Set the IP address after (isInNet(host,... to the subnet id of the your local network; for example, if your local network has IPs in the Class C range of 192.168.1.x, enter 192.168.1.0. Also change the subnet mask to match the configuration of your network. Also edit the IP address found after the word PROXY to the IP address of the proxy server. Be sure to change the port also if you used one other than 8080.

This script will be served to the web browser at the beginning of the browsing session; if a requested address is local (i.e. on your network) the proxy will be bypassed. All other requests will be handled by the proxy. This has the advantage of allowing you to use this web server for other purposes (without being affected by DansGuardian).

At this point, to test if your http server is working, you should be able to go to a web browser on your network and point it to http://x.x.x.x/wpad.dat and be prompted to download the autoconfiguration script (where x.x.x.x is the address of the web server/proxy server). If not, ensure that wpad.dat is in the root of the server (/var/www) and also that the web server has started on your server:

sudo /etc/init.d/apache2 start
OR
sudo /etc/init.d/lighttpd start

DHCP Option 1 - dhcp3-server

We will now consider the setup of dhcp3 in order to provide the WPAD information to clients automatically as part of their dhcp lease. If you already have a dhcp3 server on your network, simply follow the steps below and adapt them to suit your current configuration.

If you want to have your proxy server function as a dhcp server, please follow the steps found at https://help.ubuntu.com/community/dhcp3-server for a basic configuration that will get your box serving leases to clients. We need to make only one configuration change. Open /etc/dhcp3/dhcpd.conf for editing with nano:

sudo nano /etc/dhcp3/dhcpd.conf

...and add:

option custom-proxy-server "http://x.x.x.x/wpad.dat"

...with the other options in the file, where x.x.x.x is the IP address of the web server where you placed the WPAD script in the web server configuration; to use the Ubuntu sample configuration as an example:

default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.example";
option custom-proxy-server "http://x.x.x.x/wpad.dat";
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.10 192.168.1.100;
range 192.168.1.150 192.168.1.200;
}

Ctrl-X and "Y" to exit nano and save the changes, then do the following to restart dhcp3:

sudo /etc/init.d/dhcp3 restart

Request new dhcp leases for your clients, check "Autodetect proxy settings" in client browsers, and head to www.whatismyip.com, where you should see something like the following:

Your IP Address Is: x.x.x.x
Possible Proxy Detected: 1.0 your.fqdn.here:3128 (squid/2.7.STABLE3)

...which indicates that your client's connection is proxied!

DHCP Option 2 - Microsoft DHCP Server

Another popular option for dhcp is the Microsoft DHCP server (the corporate network for which I set up this configuration uses Windows Server 2003 and so this was my experience).

Open the DHCP configuration on Windows Server. Right click on the server and choose "Set Predefined Options...", then "Add", and in the window that pops-up, set the following options (This information has been taken from http://support.microsoft.com/default.aspx?scid=kb;EN-US;252898 ):

Name: Proxy Autoconfiguration
Data Type: String
Array (unchecked)
Code: 252
Description: Proxy Autoconfiguration Option

Hit "OK", and then in the field, String: enter the http address of the web server serving the proxy autoconfiguration script, followed by a space, e.g. "http://x.x.x.x/wpad.dat " (but without quotes). Next, right click on either "Scope Options" (if you want the proxy to only apply to a specific scope) or "Server Options" (if you want the proxy to apply globally) and choose "Configure Options", then scroll down the list to ensure that option 252 is checked (this is the option you just set up!), hit OK, and close the DHCP configuration.

Now request a new DHCP lease for your clients, set the proxy options in your web browser to autodetect configuration, and navigate to www.whatismyip.com, where you should see something like the following:

Your IP Address Is: x.x.x.x
Possible Proxy Detected: 1.0 your.fqdn.here:3128 (squid/2.7.STABLE3)

...which indicates that your connection is proxied!

Optional DNS Configuration

The WPAD setup should work without this step, but if you aren't having any luck with DHCP, set a DNS alias for your proxy server as "wpad", so that when you attempt to resolve http://wpad.your.fqdn.here/, you are redirected to the web server (possibly on your proxy server). Web browsers like Internet Explorer and Firefox look for a wpad.dat entry at this location if not properly passed the option via DHCP - various DNS servers will have different means of setting this alias.

5. (NEW!) Accessing Your Proxy Remotely With SSH

In order to be able to connect to your proxy remotely, you must (obviously) have an SSH server set up on your network. There are many ways to achieve this result. An excellent one can be found in the Ubuntu community documentation at https://help.ubuntu.com/community/SSH/OpenSSH/Configuring. Setting up this server is outside of the scope of this tutorial, but I would suggest that separating the Proxy server from the SSH server is a good idea in an environment where you don't want users circumventing the proxy. Having the two servers on the same machine means that users are free to tunnel the unprotected Squid port instead of the protected DansGuardian port. Separating the servers means that the proxy will still only allow connections to 8080. However, if this server is for home use, you are free to set it up however you wish. It is important to remember that if the SSH server and DansGuardian are on the same machine, you must set another "filterip" line in /etc/dansguardian/dansguardian.conf equal to 127.0.0.1 so that DG will listen to the localhost for proxy requests coming via SSH.

In order to use the proxy remotely, from your remote machine create an SSH tunnel to the proxy server/port:

ssh -p 22 username@ssh-server-ip -L 8080:proxy-server-ip:proxy-port -N

Where... "-p 22" is the default port of your SSH server (change it if it is different), "username" is your username on the SSH server, "ssh-server-ip" is the IP address of your remote SSH server, "proxy-server-ip" is the local IP address of your proxy server (i.e. its internal address, 192.168.1.whatever - If the SSH server and proxy server are on the same machine, use 127.0.0.1), and "proxy-port" is the... proxy port (8080 if you've been following along).

Now, you can go into a web browser on your remote machine, to proxy settings, and for the IP of the proxy server, type "localhost" OR "127.0.0.1" and for the port, 8080. Your web traffic will now be tunneled across SSH to your server, then passed to your proxy server for protected surfing!

Conclusion

One final test will ensure that anti-virus detection is working properly. Navigate to http://www.eicar.org/anti_virus_test_file.htm and scroll down the page to attempt to download the EICAR anti-virus test file, which is not malicious but should present as a threat to ClamAV. If DansGuardian is working and you haven't edited the default configuration, you should be prevented from even attempting to download several of the files because of their extensions, and should be informed as such by the DansGuardian "Access Denied" page. To change this behaviour, edit the banned lists in /etc/dansguardian/lists called "bannedextensionlist" and "bannedmimetypelist". After this you should still be prevented from downloading the Anti-Virus test file, and DansGuardian will inform you that you have been denied access due to virus content.

After having followed this tutorial, you should have a Squid caching proxy server, protected by the content filtering abilities of DansGuardian, and with virus scanning of internet files provided by ClamAV; most importantly, WPAD should ensure that the only configuration necessary on the client side is to ensure that "Autodetect proxy settings for this network" is checked in the client's web browser. And all this achieved by installing files only via apt! Hopefully this tutorial has been useful to you. If you have questions, feel free to contact me and I can help you sort through them.


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Shaker242 (not registered) on Tue, 2011-12-20 19:40.
If you want SSH access into your proxy, just add: ufw allow 22

ufw default DENY
ufw ALLOW 22
ufw ALLOW 8080
ufw DENY 3128
ufw enable

No need to try to hide ports or anything, just makes it more confusing to admin.

Submitted by BulletXt (not registered) on Mon, 2011-12-05 22:37.

Hi, I have a very important question.  In this tutorial, you explain everything and it all seems to work.  The problem is that, what's the whole point of this if a client can simply change the gateway to 192.168.1.254? Putting that gateway he will skip and bypass the whole DansGuardian + Squid rules.

 How can one be sure a client can not access to 192.168.1.254 gateway?

Submitted by Anonymous (not registered) on Mon, 2010-07-12 10:44.
Great Article! It will help me to deploy a similar box in prod. environment. Thanks
Submitted by Ankit Babbar (not registered) on Tue, 2010-02-23 13:40.
option custom-proxy-server "http://x.x.x.x/wpad.dat";

When i restart it says unknown option dhcp.custom-proxy-server

Submitted by Anonymous (not registered) on Tue, 2010-10-19 14:02.

In DHPC3 you need define option before you can use it:

option custom-proxy-server code 1 = proxy; // definition of option, number 1 is a option number in case you use more custom options
option custom-proxy-server "http://X.X.X.X/wpad.dat"; // option

and everything works

Submitted by Anonymous (not registered) on Thu, 2010-05-06 09:59.

Hi! I had similar problem,

try this:

option custom-proxy-server code 252 = "http://x.x.x.x/wpad.dat";

Submitted by cgarcia (not registered) on Wed, 2009-10-21 18:57.

I have just finished the install as outlimed in your howto but when I try to access the internet from another machine on the network this is what I get:

Unable to determine IP address from host name for www.whatismyip.org
The dnsserver returned:
Timeout
This means that:
 The cache was not able to resolve the hostname presented in the URL.
 Check if the address is correct.
Your cache administrator is webmaster.


Generated Wed, 21 Oct 2009 17:47:56 GMT by ubuntu (squid/2.7.STABLE3)
Submitted by dwains (registered user) on Wed, 2009-09-23 02:30.

I am looking to set up this very sort of thing at my home.  The part that is unclear is how the hardware is configured.  Do you need need two NICs?   Does this box need to live between the router most folks have at home and the rest of the net?  If you can describe this to me, I would be happy to draw up a diagram so that you can show this with the tutorial. 

 I was also curious how much hardware I would need to make this work.  Would an older Pentium III with 512 MB do the job?  I would imagine that you would not need any graphics for this so running at run level 3 should be fine.

Thanks

 Dwain

 

Submitted by Carlos Navarro (not registered) on Thu, 2009-08-20 15:16.

Hello People,

 I had problem setting the custom-proxy-server in dhcp3-server in a Lenny Debian distribution, I had a error.

But I solve the problem with dnsmasq:

 dhcp-option=252,http://192.168.1.1/local/wpad.dat

 Thanks for the HOWTO...

Charlie

Submitted by Anonymous (not registered) on Wed, 2009-10-21 04:01.
how to resolve it?i dont get it
Submitted by Anonymous (not registered) on Thu, 2009-07-30 22:17.

Great article, it helped me get my first linux server/proxy setup. The Win2k3 DHCP portion was also exactly what I needed.

 

Thanks!

Submitted by rob (not registered) on Mon, 2009-07-06 03:35.

I think it is odd that you threw Windows Server into the mix. The title of your article mentions Ubuntu Server. Would it not have made more sense to have an all Linux tutorial? How many Linux folks have a an expensive Windows Server around? Since I do not, I'll have to stop reading at the 'dhcp' part. I am interested in everything else you mentioned, though. Also, it would be nice to know how to surf through a home proxy using ssh and take advantage of the security when I am away.

 Rob

Submitted by Coglin (not registered) on Wed, 2009-09-09 04:15.

Hi

This in fact is exactly what I need and having an existing MS network apreciated the mix of info to suit. I have used a linux server for a dev/test environment for a long time.

Thanks for sharing for those of us in a mixed environment ( and new to having Linux in a prod environment - I am about to run an appliance just like you describe above on a virtual linux server)

Cheers

Coglin - Cousin IT

Submitted by nssy (registered user) on Wed, 2009-07-08 21:34.

Great Tutorial.

Actually Rob its an all Linux tutorial. The windows bit was just an alternative for setting dhcp.

Submitted by mtcoombes (registered user) on Wed, 2009-07-08 17:12.
@rob: Thanks for the comments; I've edited the tutorial to provide an all-Linux option for users to follow. In addition, I've provided instructions at the end for connecting to the proxy remotely via SSH for protected surfing! If you need any more help, just post!
Submitted by Jef (not registered) on Wed, 2009-12-16 09:40.

Great article! My question is this:

Can we show an intro page to the users in their browsers?
That would state there is a proxy server on the net, the name of our organization, all web access will be filtered and then "click here to continue" Would that require a script on port 80 ?

Jef

 

Submitted by madmouse (not registered) on Wed, 2010-06-02 17:50.

Adding this in the global part:

option wpad-url code 252 = text; ##(defines a new option)

and this in the scope:

 option wpad-url "http://10.0.0.17/wpad.dat"; ##(applies new option)

resolved the errors.

Submitted by Anonymous (not registered) on Thu, 2011-06-23 12:01.

Hello Sir,

I have done this setting but having problem not accessing proxy setting at client. DHCP service is running client got ip but net is not going through 8080 port . I have set global for custom-proxy-server with 252 code . But still not getting.

Submitted by Anonymous (not registered) on Thu, 2011-06-23 11:45.

Hello Sir,

I have done all this configuration in DHCP server with 252 code . But still the client cannot got proxy setting . DHCP service is running client got IP but they are not getting proxy setting.

Please help me regarding this.....