Configuring A High Availability Cluster (Heartbeat) On CentOS

This guide shows how you can set up a two node, high-availability HTTP cluster with heartbeat on CentOS. Both nodes use the Apache web server to serve the same content.

Pre-Configuration Requirements

Assign hostname node01 to primary node with IP address 172.16.4.80 to eth0.
Assign hostname node02 to slave node with IP address 172.16.4.81.

Note: on node01

uname -n

must return node01.

On node02

uname -n

must return node02.

172.16.4.82 is the virtual IP address that will be used for our Apache webserver (i.e., Apache will listen on that address).

Configuration

1. Download and install the heartbeat package. In our case we are using CentOS so we will install heartbeat with yum:

yum install heartbeat

or download these packages:

heartbeat-2.08
heartbeat-pils-2.08
heartbeat-stonith-2.08

2. Now we have to configure heartbeat on our two node cluster. We will deal with three files. These are:

authkeys
ha.cf
haresources

3. Now moving to our configuration. But there is one more thing to do, that is to copy these files to the /etc/ha.d directory. In our case we copy these files as given below:

cp /usr/share/doc/heartbeat-2.1.2/authkeys /etc/ha.d/
cp /usr/share/doc/heartbeat-2.1.2/ha.cf /etc/ha.d/
cp /usr/share/doc/heartbeat-2.1.2/haresources /etc/ha.d/

4. Now let's start configuring heartbeat. First we will deal with the authkeys file, we will use authentication method 2 (sha1). For this we will make changes in the authkeys file as below.

vi /etc/ha.d/authkeys

Then add the following lines:

auth 2
2 sha1 test-ha

Change the permission of the authkeys file:

chmod 600 /etc/ha.d/authkeys

5. Moving to our second file (ha.cf) which is the most important. So edit the ha.cf file with vi:

vi /etc/ha.d/ha.cf

Add the following lines in the ha.cf file:

logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 30
initdead 120
bcast eth0
udpport 694
auto_failback on
node node01
node node02

Note: node01 and node02 is the output generated by

uname -n

6. The final piece of work in our configuration is to edit the haresources file. This file contains the information about resources which we want to highly enable. In our case we want the webserver (httpd) highly available:

vi /etc/ha.d/haresources

Add the following line:

node01 172.16.4.82 httpd

7. Copy the /etc/ha.d/ directory from node01 to node02:

scp -r /etc/ha.d/ [email protected]:/etc/

8. As we want httpd highly enabled let's start configuring httpd:

vi /etc/httpd/conf/httpd.conf

Add this line in httpd.conf:

Listen 172.16.4.82:80

9. Copy the /etc/httpd/conf/httpd.conf file to node02:

scp /etc/httpd/conf/httpd.conf [email protected]:/etc/httpd/conf/

10. Create the file index.html on both nodes (node01 & node02):

On node01:

echo "node01 apache test server" > /var/www/html/index.html

On node02:

echo "node02 apache test server" > /var/www/html/index.html

11. Now start heartbeat on the primary node01 and slave node02:

/etc/init.d/heartbeat start

12. Open web-browser and type in the URL:

http://172.16.4.82

It will show node01 apache test server.

13. Now stop the hearbeat daemon on node01:

/etc/init.d/heartbeat stop

In your browser type in the URL http://172.16.4.82 and press enter.

It will show node02 apache test server.

14. We don't need to create a virtual network interface and assign an IP address (172.16.4.82) to it. Heartbeat will do this for you, and start the service (httpd) itself. So don't worry about this.

Don't use the IP addresses 172.16.4.80 and 172.16.4.81 for services. These addresses are used by heartbeat for communication between node01 and node02. When any of them will be used for services/resources, it will disturb hearbeat and will not work. Be carefull!!!

view as pdf | print

Share this page:

47 Comment(s)

Add comment

Comments

By: Krishna

Dear All,

I have done exactly same entries in my ha.cf file, but when I run uname -n command it does not show me both of the nodes ? Any idea how to correct it. Please help ?

By: sunny

Did you got any solution for this problem, Even I am facing same issue.

Thanks

By: cuky23

I have got this setup so that the resource group just contains IP/MASK no services, as I want the services to be running on both nodes.

I am using haproxy to route traffic to both servers from the VIP's . managed to get VIP running on each server one for WEB and one for SQL.

Only issue I have found is that if both nodes go off. It can be a bit of pain to the service heartbeat start to kick in again. in fact I have had to get the second node started before it will start heartbeat on the first node? Anyone explain why happens.

By:

Firstly, this was a great tutorial, thanks. I had trouble starting the heartbeat service using this node directive syntax in the ha.cf file

node node01
node node02

this however worked for me

node node01 node02

I am using centos 5.1 and heartbeat v2.1.3 (this might have changed since v2.1.2??) and more info on the node directive is here: http://linux-ha.org/ha.cf/NodeDirective

By: raducu

This article is really great. I followed the instruction and everything worked smoothly on centos 5.2/ Thank you very much.

By:

1st accept my appoligies for replying late. and then let me thank you for appreciating this work. Both statements have same meanings to hearbeat, perhaps i don't know why error generated.

By: Anonymous

Don't follow this manual, its confusing and sucks.

By: Anonymous

Respected Sir/Mam,

From last few days, I am working to implement High Availability Cluster with apache and heartbeat.

Resources I am using are as follow:

CentOS-5 as master-node
fedora-11 as slave-node

hearbeat 2.1.4
apache 2.2

I referred tutorial : https://www.howtoforge.com/high_availability_heartbeat_centos

At the moment, I have successfully implemented heartbeat but the problem is with apache httpd service which is not automatically getting started on heartbeat start up.

So, please guide me regarding this issue.
It would be of great help.

Reply me on [email protected]

By: Anonymous

Why say do not follow this Post, it is useful but is true missing something for Version 2, I just want to make a note for all your guy for Ref, in version 2 if you enabled "crm on" then Heartbeat will not use the "haresources" file any more and the service/virtual IP address could not create. In fact it is playing me whole night to Google search and no page saying this until I find "http://www.netexpertise.eu/en/linux/heartbeat-2-howto.html/comment-page-1#comment-15393"

By: rootlurker

Good job! Great howto's.. it's works very well.

Many thanks...

By: Webmaster

Hi There,

Everything is working but only starting HTTPd Service. I am using OpenVZ to simulate these HA. But on each node, it doesn't have eth0 interface. only venet0:0

Any ideas?

[r[email protected] ~]# netstat -untap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      28089/sendmail: acc
tcp        0      0 :::22                       :::*                        LISTEN      27998/sshd
tcp        0      0 ::ffff:192.168.150.80:22    ::ffff:192.168.150.1:55656 ESTABLISHED 1673/0

[[email protected] ~]# service httpd start
Starting httpd: (99)Cannot assign requested address: make_sock: could not bind to address 192.168.150.82:80
no listening sockets available, shutting down
Unable to open logs
[FAILED]
[[email protected] ~]#

By: DrLove73

Go to /etc/ha.d/ha.cf and use/change

"bcast eth0 eth1" to

"bcast venet0 venet1" etc.

Do not use venet0:0 or similar since heartbeat is using interfaces not IP addresses.

By:

I am facing the same issue.I try to change

bcast venet0

Bur again the same error

Heartbeat is running fine but it was not able to bring up the resource when i manually try to bring up the resource

service httpd start
Starting httpd: (99)Cannot assign requested address: make_sock: could not bind to address 192.168.1.113:80
no listening sockets available, shutting down
Unable to open logs
[FAILED]

Any help is highly appreciated

By: chr1x2

do you have "which" command?

# rpm -qa | grep which

Maybe heartbeat cannot find the gawk command. Heartbeat need the which command to find those commands on your system.

By: Deepak.

hye.!

This is a problem which may arise sometime if your virtual IP is not configured properly.

First of all ensure that you have configured virtual IP on both of ur nodes primary and secondary. here 192.168.150.80 must be the IP adder of your primary node and 192.168.0150.82 be the virtual IP.

most probably this will solve your problem.

By: Adam

the haresources file has been replaced with a XML-based cluster-wide configuration file in heartbeat v2.

Take a look here how to properly setup heartbeat.

http://www.linux-ha.org/v2/Examples/Simple

By: Carlos

Thanks for the tutorial, it worked really well.

Good Job.

By: Jesser

Excelent How-to! Works fine for me!

CentOS 5.2 x86 default install with heartbeat-2.1.3-3.el5.

Thanks!

By: Thusith

I tried with CentOS 5.3 and it works!!! Thank You.

By: chalitha

thanks man . keep it up. great work.

By: paul

Hi,

How can I configure heartbeat so that it will pass control to second node if apache in first node fails?

When I shut down node1 it switch control to node2. But if apache (only apche not heartbeat) stopped in first node, it will not pass control to second node.

Paul

By: Tony

One thing not mentioned here and something that may save someone a few hours is to make sure that there is no Firewall configured that might be blocking UDP on port 694.

When this occurs it is not apparent what is going wrong and the the HA setup immediately goes into a split brain scenario. Its quite frustrating as your installation and configuration will be correct.

A comment that I would make on the above is that for an active/active setup there is no need to configure specific resources such as httpd. This has the benefit of allowing the reaction times to be much quicker and for having services such as httpd and mysqld etc be configured for startup on reboot, irrespective of the actions of heartbeat.

Tony

By: Kensai Yanesha

I also want to thank you for this great tutorial, everything worked!

By: Anonymous

port on

172.16.4.82:80
redirects to
172.16.4.81:80
IE. 82 > 81
That is expected .

BUT
can ping the powered off box (unexpected)

Any Reason

By: Sushant Chawla

First of all thanks for this very easy to understand tutorial.

My problem is that my httpd server is not running if I am binding it on virtual ip address, it shows the error:

(99)Cannot assign requested address: make_sock: could not bind to address 192.168.15.250:80
no listening sockets available, shutting down
Unable to open logs
[FAILED]

If I enable httpd to listen on all interface heartbeat is running as expected. Please correct me if I am wrong anywhere. I have checked your howto 3-4 times & followed exactly what you have stated.

Please help...

Regds

Sushant Chawla

By:

I have properly installed and configured Heartbeat (heartbeat-2.1.3-3.el5.centos) on CentOS release 5.4. I have question as I stop heartbeat on node01 it automatically shift to node02. If i stop node02 it should automatically shift to node01 but it doesnot.
I need help regarding this(As I stop Heartbeat on Node01 it automatically shift to Node02 but vice versa it does not work, means if i stop heartbeat on Node02 it should failover to Node01, how can I do this.

Regards
SHAH JI

By: ravinder

In ha.cf file auto_failback should be on

auto_failback on

The master listed in the haresources file holds all the resources until a failover, at which time the slave takes over. When auto_failback is set to on once the master comes back online, it will take everything back from the slave. When set to off this option will prevent the master node from re-acquiring cluster resources after a failover.

By: Arn

Thanks for providing such a nice configuration doc.

As per the doc I did the configuration and it is working fine. But when i stopped httpd service on node01 HA is not able to switch to node2's httpd.

Actions taken :

step 1: Heartbeat is working fine as per your doc

step 2: stopped the httpd service on node1

step 3: http://172.16.4.82

Which is giving a black page. Why ?

-----------------------------------------------------------------------------------

Question 2:- How to configure multiple service on Heartbeat

service for httpd and mysql together. If any of the service failed it show be able to switch to next server.

By: Anonymous

I've got better results on CentOS with the Gridlock high availability cluster, see Gridlock. Yes, it's not free but I've just had it with wasting time with open source.

By: Anonymous

It actually doesnot work better with Gridlock, it is just that you have no clue what you are doing.

Try some basic linux tuts before commenting here

By:

What happened if had 1 IP WAN in this configuration?

Node in Local:
Node1: 192.168.0.1/24
Node2: 192.168.0.2/24

And IP WAN:
ISP issue: 208.67.222.222/20

By: joker400

Thanks for the great tutorial, it is really helped me

By:

that didn't work for me until I found another user who solved adding the netmask size in harerources.
In my case I used a "haresources" like this:

node01 192.168.1.92/24 httpd

By: Nitin ray

Thanks man, it's really helpful to us.

By: Avinash

getting error as below:

heartbeat[7364]: 2012/03/26_15:41:25 info: Version 2 support: false
heartbeat[7364]: 2012/03/26_15:41:25 ERROR: No heartbeat media defined
heartbeat[7364]: 2012/03/26_15:41:25 WARN: Logging daemon is disabled --enabling logging daemon is recommended
heartbeat[7364]: 2012/03/26_15:41:25 ERROR: Configuration error, heartbeat not started.

my authkeys file:

auth 2
2 sha1 test1-HA

my haresources file:

server1 192.168.225.134 httpd

my ha.cf file:

node server1
node server2

By: Sergei Petrunin

It exactly worked for me on two virtual boxes with CentOS6.2!

This possobility to stay service on line even is node is down is awesome!

I'm a noob in field of "HA" or "Clustering"... And I have little question:

What similar projects to "Heartbeat" you know? Where is the difference?

e.g.: What is the relationship between "Heatbeat" project and "Apache Hadoop" project? Is "Heartbeat" is like a part of "Hadoop" or it is something completely different?

Sergi

By: Captain.ovie

Hi,

Nice tutorial.. I have an issue, when i try starting heartbeat on node1 and node2, I see it says

[[email protected] ~]# /etc/init.d/heartbeat start

Starting High-Availability services: ERROR: Unable to find nic or netmask.

INFO: [findif] failed

INFO: Resource is stopped

Done.

[[email protected] ~]#

When I check on heartbeat status, i see it says it's running and when i check on log, I see it started. Am kind stuck here, please I need help on how to go about this.

By: postcd

Hello, please can this tutorial be used if each of my node server is in different datacenter?

By: suresh

This article is realy good !!!

By: Ayman Shorman

Thank you very much it worked with me, but i have 2 questions:

1- How does the heartbeat know that the service is down? What is using to know that the service is down or the whole machine is down?

2- Suppose both machines/services went down what is the scenario then?

Thanks

By: boby

how configure 4 server with rsync ?

please answer . . . .

By: Kuldip Singh

Dear sir,

I have completed all the steps but i am not being able to install heartbeat offline. It is giving error needs dependencies libapphb.so.2 needed by heartbeat -3.0.4-2.el6.i686 & libhbclient.so.1 needed by heartbeat -3.0.4-2.el6.i686. please tell me complete solution for installing heartbeat offline.

By: Rhyknowscerious

If you're getting a 99 error, problem binding ip addres, you might need to change a kernel parameter:

sysctl net.ipv4.ip_nonlocal_bind=1

I posted my problem on Server Fault. Here's my forum discussion: http://serverfault.com/questions/754873/apache-cant-change-listening-ip-addr

This helped me with my local Web server. Don't know if it will help a public Web server.

By: Chuong

I followed your tutorial and i used nginx service replace to httpd. When i enter address 172.16.4.82 it can't load the page, is some thing wrong ( when i go to adrress 172.16.4.80 and 172.16.4.81 , it load the default page on 2 nodes). Can you help me ?

By: walter

Hello I am new at this I would like to know if within heartbeat can set up some sort of load balancing algorithm as round robin among others ... Many thanks greetings

By: Nicholas Cutajar

Hi,

Very good totorial managed to get my quagga setup working with hearthbeat, only one question thought.

Is this setup an Active/passive setup or an Active/Active setup? (not clear for a newbie like me).

Thanks.

By: zachrison

Hi all,

I have problem when i configure and start heartbeat..

I used Ipv6..

This message error :

Starting High-Availability services: Heartbeat failure [rc=6]. Failed.

heartbeat: udpport setting must precede media statementsheartbeat[3335]: 2017/01/23_22:13:56 ERROR: glib: ucast: cannot resolve hostname

heartbeat[3335]: 2017/01/23_22:13:56 ERROR: glib: ucast: interface [eth0] does not exist

heartbeat[3335]: 2017/01/23_22:13:56 ERROR: Heartbeat not started: configuration error.

heartbeat[3335]: 2017/01/23_22:13:56 ERROR: Configuration error, heartbeat not started.

# this is my configuration /etc/ha.d/ha.cf

keepalive 2

warntime 5

deadtime 15

initdead 50

udpport 694

auto_failback on

ucast eth0 2404:6d00:1:3::18

node mail.zachrison.my.id

node mail2.zachrison.my.id

logfile /var/log/ha-log

please help ?

thanks before..

Log in or Sign up

On this page