Putting Varnish In Front Of Apache On Ubuntu/Debian

Want to support HowtoForge? Become a subscriber!
 
Submitted by Sypher (Contact Author) (Forums) on Fri, 2010-10-22 17:18. :: Debian | Ubuntu | Apache

Putting Varnish In Front Of Apache On Ubuntu/Debian

Version 1.0, 15-10-2010
Follow me on Twitter

Varnish is an open source "web accelerator" which you can use to speed up your website.

It can cache certain static elements, such as images or javascript but you can also use it for other purposes such as Loadbalancing or some additional security.

In this tutorial we will focus on the latter one.
In this mode, Varnish will stop incomplete HTTP requests from reaching your Apache webserver.

This tutorial is built on Ubuntu, but will probably also work on Debian.

First of all, make sure you are running Apache2 and have it configured.

 

Installing Varnish

This is rather easy, since it is in the Ubuntu repository. However, you might want to use the Varnish repository to make sure you have a more recent version. To add this one, execute this:

sudo curl http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add -

sudo echo "deb http://repo.varnish-cache.org/debian/ $(lsb_release -s -c) varnish-2.1" >> /etc/apt/sources.list

Update APT and install Varnish:

sudo apt-get update

sudo apt-get install varnish

Great, now you have Varnish but we still need to configure it.

 

Changing Varnish settings

First, we have to change the default port. Edit /etc/default/varnish:

vim /etc/default/varnish

Scroll down a bit, until you find an uncommented line starting with "DAEMON_OPTS".
- Change *:6081 to *:80 so it will listen on the default HTTP port.
- edit default.vcl to something else, I took "mysite.vcl".

Save the file.

Edit the VCL file you mentioned in the previous file. In my case, I'll be editing /etc/varnish/mysite.vcl. Paste the following contents:

## Redirect requests to Apache, running on port 8000 on localhost
backend apache {
        .host = "127.0.0.1";
        .port = "8000";
}
## Fetch
sub vcl_fetch {
		## Remove the X-Forwarded-For header if it exists.
        remove req.http.X-Forwarded-For;
		
		## insert the client IP address as X-Forwarded-For. This is the normal IP address of the user.
        set    req.http.X-Forwarded-For = req.http.rlnclientipaddr;
		## Added security, the "w00tw00t" attacks are pretty annoying so lets block it before it reaches our webserver
        if (req.url ~ "^/w00tw00t") {
                error 403 "Not permitted";
        }
		## Deliver the content
        return(deliver);
}

## Deliver
sub vcl_deliver {
		## We'll be hiding some headers added by Varnish. We want to make sure people are not seeing we're using Varnish.
              ## Since we're not caching (yet), why bother telling people we use it?
        remove resp.http.X-Varnish;
        remove resp.http.Via;
        remove resp.http.Age;
		
		## We'd like to hide the X-Powered-By headers. Nobody has to know we can run PHP and have version xyz of it.
        remove resp.http.X-Powered-By;
}

Save the file. All right, that was the Varnish part. Do not start it yet.

 

Changing Apache settings

OK, so we have to let Apache2 listen on localhost. For this, there are a few small changes required.

vim /etc/apache2/ports.conf

Change:

NameVirtualHost *:80
Listen 80

To:

NameVirtualHost *:8000
Listen 127.0.0.1:8000

Apache will listen on that port. You will have to edit your vhosts as well. Open your vhost(s) and replace

<VirtualHost *:80>

with

<VirtualHost *:8000>

So far so good. We now have to install an extra Apache module to make sure the IP address of the user ends up correct. Since Varnish is basically talking with Apache2, you would see 127.0.0.1 as visitor IP.

apt-get install libapache2-mod-rpaf

The RPAF (Reverse Proxy Add Forward) module will make sure the IP of 127.0.0.1 will be replaced with the IP set in X-Forwarded-For set by Varnish.

 

Restart daemons

Restart Apache:

/etc/init.d/apache2 restart

Check if it is bound to the correct IP/Port by executing:

netstat -lp | grep apache2

If you see:

tcp        0      0 localhost:8000          *:*                     LISTEN      4586/apache2

This is correct. Otherwise, you made a mistake. All right, so now we have to restart Varnish to let it listen on port 80.

/etc/init.d/varnish restart

We check this again by executing:

netstat -lp | grep varnish

The result will be:

tcp        0      0 *:www                   *:*                     LISTEN      4498/varnishd
tcp6       0      0 [::]:www                [::]:*                  LISTEN      4498/varnishd

(Yes, varnish also listens on any IPv6 address).

So.. Now we have placed Varnish in front of Apache2. We can test if the site still works by simply visiting it. You will see the site, just as nothing happened. You can test this further by shutting down apache. You will then see a Varnish error page.

 

Bonus features

Well, you might want to change the HTTP servername from "Apache" to something else. This can be done by editing your VCL file, located in /etc/varnish. After:

sub vcl_fetch {

Add:

## Remove the http.Server header
unset obj.http.Server;
## Change the http.Server header to something else
set obj.http.Server = "Incognito";

Obviously you can make it look like whatever you want. Yourdomain.com for example. Please note that all the domains on this server will use the same servername.

Well, that's all. You now have a reverse proxy in front of your Apache!
With a bit of tweaking, you can let it cache or loadbalance.


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by shobhit (not registered) on Mon, 2014-06-23 19:02.
I have a website with a database of >2 Gb. my site is just like youtube etc. but i am not hosting any video rather it is sharing youtube videos. my video pages are most surfed one. I have installed varnish on it. I find that many time varnish gives me guru mediation error. I am not able to understand how to solve this problem. Please let me know. what to do?
Submitted by Anonymous (not registered) on Fri, 2014-06-13 20:00.

I spent over a month trying to get a production server running in EC2 with Varnish and CloudFlare CDN working properly with remote client IP behind a load balancer, until I found this AMI: https://aws.amazon.com/marketplace/pp/B00KSGVT9C, it has everything pre-installed and configured, and it help me get everything up and running in a hour!

 

Submitted by Jukkis (not registered) on Mon, 2013-01-14 23:49.
Any point in using apache mod_pagespeed on this setup?
Submitted by yatint (not registered) on Thu, 2012-12-06 17:11.
Hello

I am planning to configure Varnish on my server to hide me server ip
consider my server ip is 66.66.66.66 and I want to configure varnish in such a way, when anyone tries to locate my server ip using any dns checker tool, it should show  127.0.0.1
to users. 

can you tell me how  can i do this. I tried alot and alot but no result. 

help needed.

Thanks
Yatin

Submitted by Zillur (not registered) on Fri, 2014-05-23 15:47.
Your question has actually no meaning. If you want to hide your ip , then no request come to your server. One solution ( but may be you don't mean it) using  www.cloudflare.com/ . It actually used for speeding up your website.
Submitted by Anonymous (not registered) on Thu, 2013-08-15 22:52.
Use Cloudflare.com to hide your ip address.
Submitted by Anonymous (not registered) on Thu, 2012-12-27 15:03.

If nobody knows your IP address they will not be able to connect to your varnish server - so your request makes no sense.

If you do not want people to view your website then place a firewall in front of it, to deny incoming connections.  If you do want visitors to your site then they need to lookup the IP address to connect.

Submitted by Anonymous (not registered) on Sun, 2012-01-15 02:23.

I have a centos 5.5 installation with directadmin apache and such.

I tryed using varnish infront of apache but all my websites got the message:

Apache is functioning normally

 So is it possible to forward more domains because I sell hosting to customers and it would be bad to have to configure each website through the config.

But I would like to use varnish.

What can I do?

Submitted by Torsten Zenk (not registered) on Sun, 2011-08-14 18:14.

Since Varnish 3.0 is out you might want to consider to use it on Ubuntu or Debian. The installation instructions are here:

Installation on Debian: https://www.varnish-cache.org/installation/debian
Installation on Ubuntu: https://www.varnish-cache.org/installation/ubuntu

and you can then use the ongoing information provided here.

Submitted by Franklin (not registered) on Wed, 2011-03-02 13:11.

If you get varnish failing silently (or saying "Not starting Varnish...[OK]" try starting it with less memory. On my Ubuntu VM this was causing me a problem, I started in debug mode from varnishd like this;

/etc/varnish# varnishd -f /etc/varnish/local.vcl -s malloc,1G -T 127.0.0.1:2000  -a 0.0.0.0:80 -d

by default varnish was trying to allocate 5G which my box took exception to

Submitted by itsnedkeren (registered user) on Fri, 2011-02-25 14:00.

This would be the correct syntax for removing / adding http headers:

 

 

        ## Unset http header
            unset beresp.http.Server;
       
        ## Set new http header
            set beresp.http.Server = "My Server xxxx";
 

Submitted by Lee Goile (not registered) on Thu, 2011-02-17 12:09.

It also works perfectly on Debian Squeeze using apache2.2 with multiple sites.

 

~azz

Submitted by Torsten Zenk (not registered) on Tue, 2011-02-15 16:16.

In Addition to the comment before if you still get an Error like this:

Error 503 Service Unavailable

Service Unavailable
Guru Meditation:

XID: XXXXXXX

Varnish cache server

 

try to add this in mysite.vcl on line 5:

.connect_timeout = 1s;
.first_byte_timeout = 5s;
.between_bytes_timeout = 2s;

 

This lets you controll the amount of time untill an varnish timeout happens.

Submitted by Torsten Zenk (not registered) on Tue, 2011-02-15 13:48.

If you ever get a memory locked error like the one below and varnish won't start, have a look at the mysite.vcl file:

 

myservername:/etc/varnish$ sudo /etc/init.d/varnish restart
Starting HTTP accelerator: varnishd failed!
Running VCC-compiler failed, exit 1
VCL compilation failed
storage_file: filename: /var/lib/varnish/myservername/varnish_storage.bin size 1024 MB.
Message from VCC-compiler:
Variable 'obj.http.Server' not accessible in method 'vcl_fetch'.
At: (input Line 10 Pos 7)
unset obj.http.Server;
------###############-

uncommenting line 10 and 12 in /etc/varnish/mysite.vcl solved the problem for me. Hope that helps

 

Best Regards

Torsten Zenk

Submitted by torstenzenk (not registered) on Fri, 2011-01-28 03:52.

This HowTo works like a charm on Debian Lenny. Thank you VERY much for the effort you have put into it!

 

Best Regards

Torsten