This is a "copy & paste" HowTo! The easiest way to follow this tutorial is to use a command line client/SSH client (like PuTTY for Windows) and simply copy and paste the commands (except where you have to provide own information like IP addresses, hostnames, passwords,...). This helps to avoid typos.

How To Set Up A Loadbalanced High-Availability Apache Cluster

Version 1.0
Author: Falko Timme
Last edited 04/26/2006

This tutorial shows how to set up a two-node Apache web server cluster that provides high-availability. In front of the Apache cluster we create a load balancer that splits up incoming requests between the two Apache nodes. Because we do not want the load balancer to become another "Single Point Of Failure", we must provide high-availability for the load balancer, too. Therefore our load balancer will in fact consist out of two load balancer nodes that monitor each other using heartbeat, and if one load balancer fails, the other takes over silently.

The advantage of using a load balancer compared to using round robin DNS is that it takes care of the load on the web server nodes and tries to direct requests to the node with less load, and it also takes care of connections/sessions. Many web applications (e.g. forum software, shopping carts, etc.) make use of sessions, and if you are in a session on Apache node 1, you would lose that session if suddenly node 2 served your requests. In addition to that, if one of the Apache nodes goes down, the load balancer realizes that and directs all incoming requests to the remaining node which would not be possible with round robin DNS.

For this setup, we need four nodes (two Apache nodes and two load balancer nodes) and five IP addresses: one for each node and one virtual IP address that will be shared by the load balancer nodes and used for incoming HTTP requests.

I will use the following setup here:

  • Apache node 1: webserver1.example.com (webserver1) - IP address: 192.168.0.101; Apache document root: /var/www
  • Apache node 2: webserver2.example.com (webserver2) - IP address: 192.168.0.102; Apache document root: /var/www
  • Load Balancer node 1: loadb1.example.com (loadb1) - IP address: 192.168.0.103
  • Load Balancer node 2: loadb2.example.com (loadb2) - IP address: 192.168.0.104
  • Virtual IP Address: 192.168.0.105 (used for incoming requests)

Have a look at the drawing on http://www.linuxvirtualserver.org/docs/ha/ultramonkey.html to understand how this setup looks like.

In this tutorial I will use Debian Sarge for all four nodes. I assume that you have installed a basic Debian installation on all four nodes, and that you have installed Apache on webserver1 and webserver2, with /var/www being the document root of the main web site.

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Enable IPVS On The Load Balancers

First we must enable IPVS on our load balancers. IPVS (IP Virtual Server) implements transport-layer load balancing inside the Linux kernel, so called Layer-4 switching.

loadb1/loadb2:

echo ip_vs_dh >> /etc/modules
echo ip_vs_ftp >> /etc/modules
echo ip_vs >> /etc/modules
echo ip_vs_lblc >> /etc/modules
echo ip_vs_lblcr >> /etc/modules
echo ip_vs_lc >> /etc/modules
echo ip_vs_nq >> /etc/modules
echo ip_vs_rr >> /etc/modules
echo ip_vs_sed >> /etc/modules
echo ip_vs_sh >> /etc/modules
echo ip_vs_wlc >> /etc/modules
echo ip_vs_wrr >> /etc/modules

Then we do this:

loadb1/loadb2:

modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr

If you get errors, then most probably your kernel wasn't compiled with IPVS support, and you need to compile a new kernel with IPVS support (or install a kernel image with IPVS support) now.

2 Install Ultra Monkey On The Load Balancers

Ultra Monkey is a project to create load balanced and highly available services on a local area network using Open Source components on the Linux operating system; the Ultra Monkey package provides heartbeat (used by the two load balancers to monitor each other and check if the other node is still alive) and ldirectord, the actual load balancer.

To install Ultra Monkey, we must edit /etc/apt/sources.list now and add these two lines (don't remove the other repositories):

loadb1/loadb2:

vi /etc/apt/sources.list

deb http://www.ultramonkey.org/download/3/ sarge main
deb-src http://www.ultramonkey.org/download/3 sarge main

Afterwards we do this:

loadb1/loadb2:

apt-get update

and install Ultra Monkey:

loadb1/loadb2:

apt-get install ultramonkey

If you see this warning:

  ¦ libsensors3 not functional                                               ¦
¦ ¦
¦ It appears that your kernel is not compiled with sensors support. As a ¦
¦ result, libsensors3 will not be functional on your system. ¦
¦ ¦
¦ If you want to enable it, have a look at "I2C Hardware Sensors Chip ¦
¦ support" in your kernel configuration. ¦

you can ignore it.

During the Ultra Monkey installation you will be asked a few question. Answer as follows:

Do you want to automatically load IPVS rules on boot?
<-- No

Select a daemon method.
<-- none

Share this page:

20 Comment(s)

Add comment

Comments

From: Anonymous at: 2006-04-28 08:14:56

thank you falko, it is the best guide.

Regards

noahlau

From: at: 2007-10-03 18:15:45

I must prove it in Ubuntu 6.06
 
....
 
thks.

From: John Wards at: 2008-12-17 08:21:41

Just a note to say that this tutorial works in 8.10 i386 server.

 I believe it won't work in x64.

From: Jimmy at: 2009-11-03 01:15:53

This was one of the best cluster guides I have seen. Just wanted to say thank you. We are using it on onelight.

From: Anonymous at: 2010-07-03 11:23:49

How hard would it be to set up ISPConfig in a similar fashion?

From: at: 2010-11-12 10:17:59

Tested and approved with Debian GNU/Linux 2.6.26 and backports.

 I'm wondering why there are loaded both of the scheduling "lblc" and "lblcr" ? If we have only two nodes in our cluster

 then cannot be talk for a "server set" and replication scheduling will be senseless.

 Correct me if i'm wrong.

 With respect: Imagandi

From: Anonymous at: 2011-10-24 17:24:41

Hello Everyone.. sorry if this is not the best place to ask, but I was just wondering..

Will I need a Layer4 Switch to have all this settings working?

If I do configure my nodes as above without it (switch) will I be able to still have a load-balanced Apache without it?

 Thanks so much, and appreciate the effort to make this tutorial.

See ya.

From: Anonymous at: 2011-10-25 15:04:21

Thanks for this article............

From: Anonymous at: 2014-01-10 21:11:09

Layer 3*

And no the switches "think" the ip is on both of the load balancers. It doesnt matter where it gets to since the load balancers will make sure the request gets to a node.

 

read : http://www.linuxvirtualserver.org/docs/ha/ultramonkey.html

From: Martin Sebald at: 2008-11-26 15:47:25

Hello all,

in my opinion the persistant flag is missing. As described in the introduction of this tutorial the load balancers also take care of connections/sessions.

This does NOT happen.

After I added the option "persistent" to ldirectord.cf it worked:



virtual=192.168.0.105:80
real=192.168.0.101:80 gate
real=192.168.0.102:80 gate
fallback=127.0.0.1:80 gate
service=http
request="ldirector.html"
receive="Test Page"
scheduler=rr
persistent=600
protocol=tcp
checktype=negotiate

600 means 600 seconds which also could be the default setting. In my opinion to short for stuff like forum software.

Maybe there is a easier way to handle this. Our setup is depending mostly on PHP and PHP session based stuff, so it did not work correctly before.

Kindest regards,
Martin

From: ndr at: 2009-05-27 02:13:32

I don't quite understand why this needs to be specified in a config file in order for the apache failover to occur. When I tried just creating a random file on both servers and stopped one, it didn't failover to the second apache server. Anyone know why this is?

 Your tutorial is great. I ran into problems with starting ldirector and heartbeat and it was very difficult to back up to a place where I could start fresh. i think the problem is that I had entered some IP Addresses incorrectly. Some info on how to roll out would be helpful. I did it with trial and error.

Thanks!

ndr

From: Koni at: 2010-03-09 19:11:07

Where did you configure the second ip (virtual ip) for eth0?

I made it like this:

auto eth0
iface eth0 inet static
address 192.168.0.1
netmask 255.255.255.0

#the virtual ip

auto eth0:0
iface eth0:0 inet static
address 192.168.0.100
netmask 255.255.255.0

 

does this also work?

From: Anonymous at: 2006-05-15 11:06:02

How To very mutch!!!!

By Marcos Abadi

Brazilian

From: Anonymous at: 2008-09-28 19:43:45

Hi,

seems to be a great tutorial !

I'd like to know if this kind of load balancing setup can be SSL aware, means, an SSL session is able load balanced between nodes ?

Thanks a lot 

From: Anonymous at: 2008-09-30 22:03:40

can you tell me if this tutorial implements session-aware load-balancing as we need to use this for big site which is session driven.

From: Ashish at: 2011-10-31 09:44:18

Hello,

 You can use pound as load balancers, it supports SSL feature...

 

Cheers,

Ashish Jaiswal

From: namwodahs at: 2006-10-19 21:06:06

This is a great tutorial! Worked first time through. I've been very impressed with your selection of tutorials on this site. Thank You!

From: Anonymous Coward at: 2008-11-25 11:33:43

Nice article, although I think Load Balancing is a little misleading.

This is High Availability - an active / passive server relationship, there is no load sharing

From: John Langley at: 2011-01-25 16:16:13

I know this is an old article, but it seems still valid, I hope to try it out soon with some VM's before I try this on our production site! 

That said, I saw the comment I'm replying to, and, at first glance, it seemed right and I got frustrated that I would have to continue my search on how to do HA & LB w/Apache.  But, the only portion of this solution that isn't load balanced are the balancers themselves.

As to Apache being load balanced, you can see that it is based on page 3 of the article from the output of ipvsadm -L -n.   You can see how the load balancer master is listening on the 105 virtual IP and knows about two Apache servers at 101 & 102.  Since the weights are "0" requests will be handed off 50/50 for each server (rather, new sessions will be split 50/50).  I don't know any of the commands personally yet but you'd want to change the weight based on the hardware capabilities of the servers as well as if either the 101 or 102 box had other tasks they were solely responsible for.

Also if you wanted to get real inventive I think you could set your session directory to be on a SAN, so that if the 101 or 102 box went down, the sessions wouldn't be blown away ... something I hope to test!

From: faizan at: 2009-11-12 06:20:49

hey guys i have done almost complete work and all tests went fine but the only problem which i am facing is in ldirectord.cf file ..........................my request is not forwarding toward the real server......................

if anybody knows about the problem plz help me .........