Virtual Machine Replication & Failover with VMWare Server & Debian Etch (4.0) - Page 3

4. Configuring the Heartbeat package

Installing the Heartbeat package will enable automatic failover and resource management in case of resource and/or hardware failure.

In this tutorial, I will use Heartbeat over the network, but it is also possible to use Heartbeat with a serial cable and other methods. Heartbeat will check for updates from both servers, and when not receiving any more updates will initiate a failover and takeover of the specified resources (in our case, DRBD and VMWare).

UNLESS OTHERWISE SPECIFIED, ALL COMMANDS ARE DONE ON BOTH SERVERS

OK, on to the configuration. For your convenience, you can use the following configuration files as they are tailored to this tutorial. Heartbeat needs 3 files to be configured, namely: "ha.cf", "haresources" and "authkeys". Edit them accordingly with your favorite editor:

nano /etc/ha.d/ha.cf

logfile	/var/log/ha-log
logfacility	local0
keepalive 1
deadtime 10
warntime 10
udpport	694
bcast	eth1
auto_failback off
node	server1  ## make sure both names are accessible - check /etc/hosts
node	server2
ping 192.168.0.254  ## Enter an IP address that is pingable from the ETH0 network !!
respawn hacluster /usr/lib/heartbeat/ipfail

NOTE: This ha.cf file will enable failover in case of Network Connection failure on ETH0 and/or general server failure. Problems with ETH1 will NOT cause a failover to occur !

Now we edit the "authkeys" file:

nano /etc/ha.d/authkeys

auth 3
3 md5 failover  ## this is just a string, enter what you want ! auth 3 md5 uses md5 encryption

But we also have to chmod this file to only be readable by "root":

chmod 600 /etc/ha.d/authkeys

And for last, we edit the "haresources" file that contains the failover resources.

nano /etc/ha.d/haresources

server1 192.168.0.10 drbddisk::vm1 Filesystem::/dev/drbd0::/var/vm::ext3 vmstart
# Explanation:
# Primary Server name --> virtual IP address to be used --> DRBD resource as configurd in /etc/drbd.conf
# --> where to mount the DRBD resource and the filesystem type --> resource to start/stop in case of failover
# the "vmstart" resource is a custom script needed for VMWare Server since you can't have heartbeat take control of
# the vmware services or you will run into the dreaded "not configured for this machine" errors !

Before going further, we will need to create the custom "vmstart" script to make sure that Virtual Machines are automatically started on the server in case of failover. With other resources like Apache, Mysql, NFS ... we could have simply removed the startup scripts for them from the rc.d directories, but with VMWare this is not possible as they are required for good functioning of the services. Therefore, we need a custom script to start the Virtual Machines in case of failover since the VMWare services will already be running prior to failover !

The "vmstart" will need to reside in the "/etc/ha.d/resource.d/" folder as that is where heartbeat will check for it.

nano /etc/ha.d/resource.d/vmstart

#!/bin/bash 
case "$1" in 
start) 
/usr/bin/vmware-cmd -s register "/var/vm/'virtual machine name'/'vm config file'.vmx" 2>/dev/null
/usr/bin/vmware-cmd "/var/vm/'virtual machine name'/'vm config file'.vmx" start 2>/dev/null 
;;
stop) 
/usr/bin/vmware-cmd "/var/vm/'virtual machine name'/'vm config file'.vmx" stop trysoft 2> /dev/null 
;;
status) 
if `/usr/bin/vmware-cmd "/var/vm/'virtual machine name'/'vm config file'.vmx" getstate 2>/dev/null | grep -q "getstate() = on"` 
then 
echo "running" 
else 
echo "stopped" 
fi
;;
*) 
echo "Usage: `basename $0` 'virtual machine file' {start|stop|status}" 
echo "Where "virtual machine file" is like /var/vm/'virtual machine name'/'vm config file'.vmx"
;;
esac 
exit 0

Fill in the 'virtual machine' name and 'vm config file' directives with the necessary info (check the /var/vm dir). If you have multiple Virtual Machines, just copy the lines.

This custom script will first Register the Virtual Machine (necessary when using a newly installed server!) and then start them. It has to be done this way, since VMWare Server only has the option to automatically start VM's on system startup.
Now that you have configured everything, it is time to test if all is OK. Start the Heartbeat service, first on Server1, then on Server2.

/etc/init.d/heartbeat start

If all is OK, you should see the new virtual IP showing up on your master server.

ifconfig |more

Output:

eth0 Link encap:Ethernet  HWaddr 00:11:09:00:BB:5D
          inet addr:192.168.0.1  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::211:9ff:fe00:bb5d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1025774 errors:0 dropped:0 overruns:0 frame:0
          TX packets:227653 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:281072399 (268.0 MiB)  TX bytes:56403081 (53.7 MiB)
eth0:0    Link encap:Ethernet  HWaddr 00:11:09:00:BB:5D
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          ...

Ok, if you are feeling lucky and have followed this tutorial with some common sense, then it is now time to yank out that power cord, hold on to your cup of coffee and watch the Virtual Machine go online on your second server.