Squid Proxy Server On Ubuntu 9.04 Server With DansGuardian, ClamAV, And WPAD (Proxy Auto-Detection) [Updated 07-08-09]

Squid Proxy Server on Ubuntu 9.04 with DansGuardian, ClamAV, and WPAD (Proxy Auto-Detection)

Version 1.1
Mark Coombes (mtcoombes at gmail dot com)

Changelog:
1.0 - First version of this tutorial
1.1 - Added information on achieving WPAD through dhcp3; added section on lighttpd; added information on SSH

This tutorial will demonstrate how to set up a Squid Proxy server on Ubuntu 9.04 with DansGuardian (for content filtering) and ClamAV (for Virus scanning); in addition, we will set up Web Proxy AutoDetection (WPAD) through DHCP (in this case, either dhcp3 OR the Windows Server 2003 DHCP server) or DNS so that the only configuration necessary on the client side is to check "Auto-detect proxy settings for this network" in Firefox or Internet Explorer. At the end of the tutorial, users will have a fully functional and secure proxy for HTTP access.

In addition... the final section will show users how to connect to this proxy server via SSH in order to have a protected browsing session while away from the home/office.

NOTE: This tutorial does NOT create a transparent proxy! Internet traffic other than that passed through a web browser will not be proxied! These are the steps that worked for me, but I make no claim that they will work for you without modification.

Preliminary Configuration

This tutorial assumes that you have already installed Ubuntu Server 9.04, updated and upgraded installed packages to their latest versions. Preconfiguration of Squid, DansGuardian, ClamAV, dhcp3, and Apache or Lighttpd is not necessary - the following steps will show you all the configuration that is necessary. All installation and configuration must be done as a user with Sudo privileges.

1. Installing and Configuring ClamAV

Execute the following from a shell to install ClamAV:

sudo apt-get install clamav-daemon clamav-freshclam

This will install the ClamAV daemon necessary for virus scanning with DansGuardian, and also FreshClam to ensure that the virus database is always kept up-to-date. You may get the following warning:

LibClamAV Warning:***********************************************************
LibClamAV Warning: *** This version of the ClamAV engine is outdated.***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***
LibClamAV Warning: ***********************************************************

Which indicates that the virus engine itself is out-of-date because of the version packaged with Ubuntu 9.04. For the highest security, you should endeavour to keep the ClamAV engine updated to the latest version, but this is not possible through apt and so is out of the scope of this guide. For the most part, I believe that this warning can be ignored.

The configuration files for ClamAV are located in /etc/clamav, but the default configuration is appropriate for our needs. FreshClam will update virus definitions hourly; if you want to change this behaviour, edit /etc/clamav/freshclam.conf and change the section:

# Check for new database 24 times a day
Checks 24

To the number of times you want to check for updates, e.g.:

# Check for new database 24 times a day
Checks 48

To do it twice as often. Press Ctrl+X to exit and answer Y when nano asks if you want to "Save the Modified Buffer." If you have changed the configuration of FreshClam, execute the following to ensure the new configuration is loaded:

sudo /etc/init.d/clamav-freshclam restart

2. Installing and Configuring Squid

Execute the following from a shell to install Squid:

sudo apt-get install squid

The configuration directives for Squid are located in /etc/squid/squid.conf. Edit this file now by typing:

sudo nano /etc/squid/squid.conf

We do not have to do much to the default configuration of Squid because DansGuardian will sit in-between Squid and the users and direct traffic to the appropriate Squid port. Set this port now by adding this parameter to the bottom of the file:

http_port 3128

Press Ctrl+X to exit and answer Y when nano asks if you want to "Save the Modified Buffer." Then type the following to have Squid reload its configuration file:

sudo /etc/init.d/squid reload

3. Installing and Configuring DansGuardian

Execute the following from a shell to install DansGuardian:

sudo apt-get install dansguardian

The configuration files for DansGuardian are located in /etc/dansguardian. Edit the main configuration now by typing:

sudo nano /etc/dansguardian/dansguardian.conf

The entire dansguardian.conf is well commented and informative. The following will demonstrate a basic set-up, but to really get DansGuardian's content-filtering abilities working for you, you should read this file thoroughly. By default, DansGuardian is very restrictive (which is likely what you are after); relaxing these restrictions involves some patience and editing of configuration files. I will touch on some of this below. Take note of the following lines in the configuration:

language = 'ukenglish'
filterip = x.x.x.x
filterport = 8080
proxyip = 127.0.0.1
proxyport = 3128
#contentscanner = '/etc/dansguardian/contentscanners/clamav.conf'

The "language" directive indicates the language of the error page DansGuardian will display when it blocks a web page. Possible values for this option are found in the /etc/dansguardian directory. Once you have decided upon the language of the error page you would like to display to your users, change the language string in /etc/dansguardian/dansguardian.conf; and afterwards customize the error page:

sudo nano /etc/dansguardian/languages/ukenglish/template.html

Follow the instructions in template.html to insert your company name, and to customize the message for your purposes.

The "filterip" and "filterport" directives define, respectively, the ip address and port of the proxy server. Set filterip to equal the ip that this machine has on your LAN, and filterport equal to the port you want the proxy to have; a good default is 8080.

Important: If you want to be able to access this proxy remotely via an SSH server on the same machine, add another "filterip" line, and use the IP 127.0.0.1 - this will ensure that DansGuardian will also listen to localhost also for connection requests.

The "proxyip" directive is the IP address of the Squid proxy and "proxyport" is the port that the proxy listens on. If you have been following this guide, leave it at 127.0.0.1, in other words, the local machine. Similarly, if you used the default Squid port suggested earlier, you can leave proxyport as 3128.

The "contentscanner" line is already in /etc/dansguardian/dansguardian.conf towards the bottom. Uncomment this line to tell DansGuardian to use ClamAV to scan items requested via HTTP.

Finally, comment out or delete the line:

UNCONFIGURED - Please remove this line after configuration

To tell DansGuardian that you have changed the default configuration. Save this file using Ctrl+X and "Y", then do:

sudo /etc/init.d/dansguardian restart

To have DansGuardian reread its configuration and connect to the running Squid proxy.

At this point, do the following to secure the Squid proxy and prevent users from circumventing DansGuardian:
NOTE: Do NOT do this if you have other firewall rules in place! This will mess them up. Simply adapt whatever other firewall method you are using to take into account the following.

sudo ufw default DENY
sudo ufw ALLOW 8080
sudo ufw enable

This ensures that all incoming connections other than to port 8080 (the DansGuardian proxy port) are blocked (preventing sneaky users from accessing the open Squid proxy on port 3128). Alternately, you could explicitly block 3128 with:

sudo ufw DENY 3128

But I chose the first way to make the system more secure overall.

The directory /etc/dansguardian/lists contains all of the files related to content-filtering. Read through each of these files carefully to get an idea of the shaping possible with DansGuardian, and also to relax or tighten the strictness of DansGuardian's defaults. Since we are using ClamAV in conjunction with DansGuardian, a directory worth checking out is /etc/dansguardian/lists/contentscanners, which contains exception lists for the files that you don't want to be checked by ClamAV; several exceptions are listed in these files by default, so they are worth checking to ensure your needs are met!

4. Setting up Web Proxy Auto Detection (WPAD)

At this point, if everything has succeeded, you should be able to go into the proxy settings of your web browser and input the address and port of the proxy server we have just set up, and have a proxied connection protected by DansGuardian and ClamAV. We want this information to be provided automatically for users, so we must proceed with the following steps to set up Auto Detection; note that this procedure still requires you to set up the web browser initially (i.e., tick the "Automatically detect settings" box). For a more 'transparent' way to set up a proxy, you will need to look into Transparent proxying with Squid, but this is out of the scope of the current tutorial.

Essentially, WPAD works by serving a script to the browsers on your network via an http server. Since I want this machine to be an all-in-one solution, I will go ahead with installing a web server now (no fancy configuration is necessary, since users will not be interacting with the web server directly). If you have a web server already existing on your network, and clients can connect to it locally, you can also adapt this configuration to place the WPAD script on that server instead.

Web Server Option 1 - Apache2

First, we install Apache:

sudo apt-get install apache2

We need to add one line to /etc/apache2/httpd.conf; do so by executing:

sudo nano /etc/apache2/httpd.conf

And add the following line to the file (this associates the correct mime type for the autoconfiguration script to the .dat extension):

AddType application/x-ns-proxy-autoconfig .dat

Exit and save with Ctrl+X and "Y", then reload (or if Apache hasn't started, start it) with:

sudo /etc/init.d/apache2 reload
OR
sudo /etc/init.d/apache2 start

Now in a web browser navigate to the IP address of your server; a default Apache configuration should return "It Works!" which will let you know that the server is working!

Web Server Option 2 - lighttpd

The ideal web server for this application (if you won't be needing it for anything else) would appear to be lighttpd. I haven't tested lighttpd extensively, but the following section will help you to install and configure it to serve the WPAD script to clients. First, install lighttpd:

sudo apt-get install lighttpd

We need to associate the proper mime-type to the configuration script. To do so we must edit /etc/lighttpd/lighttpd.conf:

sudo nano /etc/lighttpd/lighttpd.conf

Add the following section to the bottom of the file (a default install should not have an existing mime-type section):

mimetype.assign = ( ".dat" => "application/x-ns-proxy-autoconfig" )

Ctrl-X and "Y" to exit nano and save the changes, then do:

sudo /etc/init.d/lighttpd restart

To restart lighttpd. Now in a web browser navigate to the IP address of your server and you should be greeted with the lighttpd default page, letting you know that the server is working!

Creating the Autoconfiguration Script "wpad.dat"

By default, the root of the web server (for either case) is in /var/www. Do the following to create the autoconfiguration script:

sudo touch /var/www/wpad.dat
sudo nano /var/www/wpad.dat

And paste the following into the file:

function FindProxyForURL(url, host)
{
if (isInNet(host, "x.x.x.0", "255.255.255.0"))
return "DIRECT";
else
return "PROXY x.x.x.x:8080";
}

There are many ways to create such a script, to achieve different purposes. The above script will need to be edited to match your configuration. Set the IP address after (isInNet(host,... to the subnet id of the your local network; for example, if your local network has IPs in the Class C range of 192.168.1.x, enter 192.168.1.0. Also change the subnet mask to match the configuration of your network. Also edit the IP address found after the word PROXY to the IP address of the proxy server. Be sure to change the port also if you used one other than 8080.

This script will be served to the web browser at the beginning of the browsing session; if a requested address is local (i.e. on your network) the proxy will be bypassed. All other requests will be handled by the proxy. This has the advantage of allowing you to use this web server for other purposes (without being affected by DansGuardian).

At this point, to test if your http server is working, you should be able to go to a web browser on your network and point it to http://x.x.x.x/wpad.dat and be prompted to download the autoconfiguration script (where x.x.x.x is the address of the web server/proxy server). If not, ensure that wpad.dat is in the root of the server (/var/www) and also that the web server has started on your server:

sudo /etc/init.d/apache2 start
OR
sudo /etc/init.d/lighttpd start

DHCP Option 1 - dhcp3-server

We will now consider the setup of dhcp3 in order to provide the WPAD information to clients automatically as part of their dhcp lease. If you already have a dhcp3 server on your network, simply follow the steps below and adapt them to suit your current configuration.

If you want to have your proxy server function as a dhcp server, please follow the steps found at https://help.ubuntu.com/community/dhcp3-server for a basic configuration that will get your box serving leases to clients. We need to make only one configuration change. Open /etc/dhcp3/dhcpd.conf for editing with nano:

sudo nano /etc/dhcp3/dhcpd.conf

...and add:

option custom-proxy-server "http://x.x.x.x/wpad.dat"

...with the other options in the file, where x.x.x.x is the IP address of the web server where you placed the WPAD script in the web server configuration; to use the Ubuntu sample configuration as an example:

default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.example";
option custom-proxy-server "http://x.x.x.x/wpad.dat";
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.10 192.168.1.100;
range 192.168.1.150 192.168.1.200;
}

Ctrl-X and "Y" to exit nano and save the changes, then do the following to restart dhcp3:

sudo /etc/init.d/dhcp3 restart

Request new dhcp leases for your clients, check "Autodetect proxy settings" in client browsers, and head to www.whatismyip.com, where you should see something like the following:

Your IP Address Is: x.x.x.x
Possible Proxy Detected: 1.0 your.fqdn.here:3128 (squid/2.7.STABLE3)

...which indicates that your client's connection is proxied!

DHCP Option 2 - Microsoft DHCP Server

Another popular option for dhcp is the Microsoft DHCP server (the corporate network for which I set up this configuration uses Windows Server 2003 and so this was my experience).

Open the DHCP configuration on Windows Server. Right click on the server and choose "Set Predefined Options...", then "Add", and in the window that pops-up, set the following options (This information has been taken from http://support.microsoft.com/default.aspx?scid=kb;EN-US;252898 ):

Name: Proxy Autoconfiguration
Data Type: String
Array (unchecked)
Code: 252
Description: Proxy Autoconfiguration Option

Hit "OK", and then in the field, String: enter the http address of the web server serving the proxy autoconfiguration script, followed by a space, e.g. "http://x.x.x.x/wpad.dat " (but without quotes). Next, right click on either "Scope Options" (if you want the proxy to only apply to a specific scope) or "Server Options" (if you want the proxy to apply globally) and choose "Configure Options", then scroll down the list to ensure that option 252 is checked (this is the option you just set up!), hit OK, and close the DHCP configuration.

Now request a new DHCP lease for your clients, set the proxy options in your web browser to autodetect configuration, and navigate to www.whatismyip.com, where you should see something like the following:

Your IP Address Is: x.x.x.x
Possible Proxy Detected: 1.0 your.fqdn.here:3128 (squid/2.7.STABLE3)

...which indicates that your connection is proxied!

Optional DNS Configuration

The WPAD setup should work without this step, but if you aren't having any luck with DHCP, set a DNS alias for your proxy server as "wpad", so that when you attempt to resolve http://wpad.your.fqdn.here/, you are redirected to the web server (possibly on your proxy server). Web browsers like Internet Explorer and Firefox look for a wpad.dat entry at this location if not properly passed the option via DHCP - various DNS servers will have different means of setting this alias.

5. (NEW!) Accessing Your Proxy Remotely With SSH

In order to be able to connect to your proxy remotely, you must (obviously) have an SSH server set up on your network. There are many ways to achieve this result. An excellent one can be found in the Ubuntu community documentation at https://help.ubuntu.com/community/SSH/OpenSSH/Configuring. Setting up this server is outside of the scope of this tutorial, but I would suggest that separating the Proxy server from the SSH server is a good idea in an environment where you don't want users circumventing the proxy. Having the two servers on the same machine means that users are free to tunnel the unprotected Squid port instead of the protected DansGuardian port. Separating the servers means that the proxy will still only allow connections to 8080. However, if this server is for home use, you are free to set it up however you wish. It is important to remember that if the SSH server and DansGuardian are on the same machine, you must set another "filterip" line in /etc/dansguardian/dansguardian.conf equal to 127.0.0.1 so that DG will listen to the localhost for proxy requests coming via SSH.

In order to use the proxy remotely, from your remote machine create an SSH tunnel to the proxy server/port:

ssh -p 22 username@ssh-server-ip -L 8080:proxy-server-ip:proxy-port -N

Where... "-p 22" is the default port of your SSH server (change it if it is different), "username" is your username on the SSH server, "ssh-server-ip" is the IP address of your remote SSH server, "proxy-server-ip" is the local IP address of your proxy server (i.e. its internal address, 192.168.1.whatever - If the SSH server and proxy server are on the same machine, use 127.0.0.1), and "proxy-port" is the... proxy port (8080 if you've been following along).

Now, you can go into a web browser on your remote machine, to proxy settings, and for the IP of the proxy server, type "localhost" OR "127.0.0.1" and for the port, 8080. Your web traffic will now be tunneled across SSH to your server, then passed to your proxy server for protected surfing!

Conclusion

One final test will ensure that anti-virus detection is working properly. Navigate to http://www.eicar.org/anti_virus_test_file.htm and scroll down the page to attempt to download the EICAR anti-virus test file, which is not malicious but should present as a threat to ClamAV. If DansGuardian is working and you haven't edited the default configuration, you should be prevented from even attempting to download several of the files because of their extensions, and should be informed as such by the DansGuardian "Access Denied" page. To change this behaviour, edit the banned lists in /etc/dansguardian/lists called "bannedextensionlist" and "bannedmimetypelist". After this you should still be prevented from downloading the Anti-Virus test file, and DansGuardian will inform you that you have been denied access due to virus content.

After having followed this tutorial, you should have a Squid caching proxy server, protected by the content filtering abilities of DansGuardian, and with virus scanning of internet files provided by ClamAV; most importantly, WPAD should ensure that the only configuration necessary on the client side is to ensure that "Autodetect proxy settings for this network" is checked in the client's web browser. And all this achieved by installing files only via apt! Hopefully this tutorial has been useful to you. If you have questions, feel free to contact me and I can help you sort through them.