Putting Varnish In Front Of Apache On Ubuntu/Debian
Version 1.0, 15-10-2010
Follow me on Twitter
Varnish is an open source "web accelerator" which you can use to speed up your website.
It can cache certain static elements, such as images or javascript but you can also use it for other purposes such as Loadbalancing or some additional security.
In this tutorial we will focus on the latter one.
In this mode, Varnish will stop incomplete HTTP requests from reaching your Apache webserver.
This tutorial is built on Ubuntu, but will probably also work on Debian.
First of all, make sure you are running Apache2 and have it configured.
Installing Varnish
This is rather easy, since it is in the Ubuntu repository. However, you might want to use the Varnish repository to make sure you have a more recent version. To add this one, execute this:
sudo curl http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add -
sudo echo "deb http://repo.varnish-cache.org/debian/ $(lsb_release -s -c) varnish-2.1" >> /etc/apt/sources.list
Update APT and install Varnish:
sudo apt-get update
sudo apt-get install varnish
Great, now you have Varnish but we still need to configure it.
Changing Varnish settings
First, we have to change the default port. Edit /etc/default/varnish:
vim /etc/default/varnish
Scroll down a bit, until you find an uncommented line starting with "DAEMON_OPTS".
- Change *:6081 to *:80 so it will listen on the default HTTP port.
- edit default.vcl to something else, I took "mysite.vcl".
Save the file.
Edit the VCL file you mentioned in the previous file. In my case, I'll be editing /etc/varnish/mysite.vcl. Paste the following contents:
## Redirect requests to Apache, running on port 8000 on localhost
backend apache {
.host = "127.0.0.1";
.port = "8000";
}
## Fetch
sub vcl_fetch {
## Remove the X-Forwarded-For header if it exists.
remove req.http.X-Forwarded-For;
## insert the client IP address as X-Forwarded-For. This is the normal IP address of the user.
set req.http.X-Forwarded-For = req.http.rlnclientipaddr;
## Added security, the "w00tw00t" attacks are pretty annoying so lets block it before it reaches our webserver
if (req.url ~ "^/w00tw00t") {
error 403 "Not permitted";
}
## Deliver the content
return(deliver);
}
## Deliver
sub vcl_deliver {
## We'll be hiding some headers added by Varnish. We want to make sure people are not seeing we're using Varnish.
## Since we're not caching (yet), why bother telling people we use it?
remove resp.http.X-Varnish;
remove resp.http.Via;
remove resp.http.Age;
## We'd like to hide the X-Powered-By headers. Nobody has to know we can run PHP and have version xyz of it.
remove resp.http.X-Powered-By;
}
Save the file. All right, that was the Varnish part. Do not start it yet.
Changing Apache settings
OK, so we have to let Apache2 listen on localhost. For this, there are a few small changes required.
vim /etc/apache2/ports.conf
Change:
NameVirtualHost *:80 Listen 80
To:
NameVirtualHost *:8000 Listen 127.0.0.1:8000
Apache will listen on that port. You will have to edit your vhosts as well. Open your vhost(s) and replace
<VirtualHost *:80>
with
<VirtualHost *:8000>
So far so good. We now have to install an extra Apache module to make sure the IP address of the user ends up correct. Since Varnish is basically talking with Apache2, you would see 127.0.0.1 as visitor IP.
apt-get install libapache2-mod-rpaf
The RPAF (Reverse Proxy Add Forward) module will make sure the IP of 127.0.0.1 will be replaced with the IP set in X-Forwarded-For set by Varnish.
Restart daemons
Restart Apache:
/etc/init.d/apache2 restart
Check if it is bound to the correct IP/Port by executing:
netstat -lp | grep apache2
If you see:
tcp 0 0 localhost:8000 *:* LISTEN 4586/apache2
This is correct. Otherwise, you made a mistake. All right, so now we have to restart Varnish to let it listen on port 80.
/etc/init.d/varnish restart
We check this again by executing:
netstat -lp | grep varnish
The result will be:
tcp 0 0 *:www *:* LISTEN 4498/varnishd
tcp6 0 0 [::]:www [::]:* LISTEN 4498/varnishd
(Yes, varnish also listens on any IPv6 address).
So.. Now we have placed Varnish in front of Apache2. We can test if the site still works by simply visiting it. You will see the site, just as nothing happened. You can test this further by shutting down apache. You will then see a Varnish error page.
Bonus features
Well, you might want to change the HTTP servername from "Apache" to something else. This can be done by editing your VCL file, located in /etc/varnish. After:
sub vcl_fetch {
Add:
## Remove the http.Server header unset obj.http.Server; ## Change the http.Server header to something else set obj.http.Server = "Incognito";
Obviously you can make it look like whatever you want. Yourdomain.com for example. Please note that all the domains on this server will use the same servername.
Well, that's all. You now have a reverse proxy in front of your Apache!
With a bit of tweaking, you can let it cache or loadbalance.
16 Comment(s)
Comments
This HowTo works like a charm on Debian Lenny. Thank you VERY much for the effort you have put into it!
Best Regards
Torsten
If you ever get a memory locked error like the one below and varnish won't start, have a look at the mysite.vcl file:
myservername:/etc/varnish$ sudo /etc/init.d/varnish restart
Starting HTTP accelerator: varnishd failed!
Running VCC-compiler failed, exit 1
VCL compilation failed
storage_file: filename: /var/lib/varnish/myservername/varnish_storage.bin size 1024 MB.
Message from VCC-compiler:
Variable 'obj.http.Server' not accessible in method 'vcl_fetch'.
At: (input Line 10 Pos 7)
unset obj.http.Server;
------###############-
uncommenting line 10 and 12 in /etc/varnish/mysite.vcl solved the problem for me. Hope that helps
Best Regards
Torsten Zenk
In Addition to the comment before if you still get an Error like this:
Error 503 Service Unavailable
Service Unavailable
Guru Meditation:
XID: XXXXXXX
Varnish cache server
try to add this in mysite.vcl on line 5:
.connect_timeout = 1s;
.first_byte_timeout = 5s;
.between_bytes_timeout = 2s;
This lets you controll the amount of time untill an varnish timeout happens.
It also works perfectly on Debian Squeeze using apache2.2 with multiple sites.
~azz
This would be the correct syntax for removing / adding http headers:
## Unset http header
unset beresp.http.Server;
## Set new http header
set beresp.http.Server = "My Server xxxx";
If you get varnish failing silently (or saying "Not starting Varnish...[OK]" try starting it with less memory. On my Ubuntu VM this was causing me a problem, I started in debug mode from varnishd like this;
/etc/varnish# varnishd -f /etc/varnish/local.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:80 -d
by default varnish was trying to allocate 5G which my box took exception to
Since Varnish 3.0 is out you might want to consider to use it on Ubuntu or Debian. The installation instructions are here:
Installation on Debian: https://www.varnish-cache.org/installation/debian
Installation on Ubuntu: https://www.varnish-cache.org/installation/ubuntu
and you can then use the ongoing information provided here.
I have a centos 5.5 installation with directadmin apache and such.
I tryed using varnish infront of apache but all my websites got the message:
Apache is functioning normally
So is it possible to forward more domains because I sell hosting to customers and it would be bad to have to configure each website through the config.
But I would like to use varnish.
What can I do?
If nobody knows your IP address they will not be able to connect to your varnish server - so your request makes no sense.
If you do not want people to view your website then place a firewall in front of it, to deny incoming connections. If you do want visitors to your site then they need to lookup the IP address to connect.
Use Cloudflare.com to hide your ip address.
Your question has actually no meaning. If you want to hide your ip , then no request come to your server. One solution ( but may be you don't mean it) using www.cloudflare.com/ . It actually used for speeding up your website.
Any point in using apache mod_pagespeed on this setup?
I spent over a month trying to get a production server running in EC2 with Varnish and CloudFlare CDN working properly with remote client IP behind a load balancer, until I found this AMI: https://aws.amazon.com/marketplace/pp/B00KSGVT9C, it has everything pre-installed and configured, and it help me get everything up and running in a hour!
I have a website with a database of >2 Gb. my site is just like youtube etc. but i am not hosting any video rather it is sharing youtube videos. my video pages are most surfed one. I have installed varnish on it. I find that many time varnish gives me guru mediation error. I am not able to understand how to solve this problem. Please let me know. what to do?
I'm migriting from Debian Wheezy to Jessie. Currently gtmetrix com site (checking form London location) shows 94/91 points for my ebinxdom pl site. I am curently using APC, memcache, but I wonder if varnish could help to achive better performance. I'm after faster server response time which I've been struggling for sometime now (e.g. now I only get 0.30-0.40ms resonse time)
English |
Deutsch