This is a "copy & paste" HowTo! The easiest way to follow this tutorial is to use a command line client/SSH client (like PuTTY for Windows) and simply copy and paste the commands (except where you have to provide own information like IP addresses, hostnames, passwords,...). This helps to avoid typos.

Setting Up A Highly Available NFS Server

Version 1.0
Author: Falko Timme
Last edited: 03/07/2006

In this tutorial I will describe how to set up a highly available NFS server that can be used as storage solution for other high-availability services like, for example, a cluster of web servers that are being loadbalanced. If you have a web server cluster with two or more nodes that serve the same web site(s), than these nodes must access the same pool of data so that every node serves the same data, no matter if the loadbalancer directs the user to node 1 or node n. This can be achieved with an NFS share on an NFS server that all web server nodes (the NFS clients) can access.

As we do not want the NFS server to become another "Single Point of Failure", we have to make it highly available. In fact, in this tutorial I will create two NFS servers that mirror their data to each other in realtime using DRBD and that monitor each other using heartbeat, and if one NFS server fails, the other takes over silently. To the outside (e.g. the web server nodes) these two NFS servers will appear as a single NFS server.

In this setup I will use Debian Sarge (3.1) for the two NFS servers as well as for the NFS client (which represents a node of the web server cluster).

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 My Setup

In this document I use the following systems:

  • NFS server 1: server1.example.com, IP address: 192.168.0.172; I will refer to this one as server1.
  • NFS server 2: server2.example.com, IP address: 192.168.0.173; I will refer to this one as server2.
  • Virtual IP address: I use 192.168.0.174 as the virtual IP address that represents the NFS cluster to the outside.
  • NFS client (e.g. a node from the web server cluster): client.example.com, IP address: 192.168.0.100; I will refer to the NFS client as client.
  • The /data directory will be mirrored by DRBD between server1 and server2. It will contain the NFS share /data/export.

2 Basic Installation Of server1 and server2

First we set up two basic Debian systems for server1 and server2. You can do it as outlined on the first two pages of this tutorial: http://www.howtoforge.com/perfect_setup_debian_sarge. As hostname, you enter server1 and server2 respectively, and as domain you enter example.com.

Regarding the partitioning, I use the following partition scheme:

/dev/sda1 -- 100 MB /boot (primary, ext3, Bootable flag: on)
/dev/sda5 -- 5000 MB / (logical, ext3)
/dev/sda6 -- 1000 MB swap (logical)
/dev/sda7 -- 150 MB unmounted (logical, ext3)
(will contain DRBD's meta data)
/dev/sda8 -- 26 GB unmounted (logical, ext3)
(will contain the /data directory)

You can vary the sizes of the partitions depending on your hard disk size, and the names of your partition might also vary, depending on your hardware (e.g. you might have /dev/hda1 instead of /dev/sda1 and so on). However, it is important that /dev/sda7 has a little more than 128 MB because we will use this partition for DRBD's meta data which uses 128 MB. Also, make sure /dev/sda7 as well as /dev/sda8 are identical in size on server1 and server2, and please do not mount them (when the installer asks you:

No mount point is assigned for the ext3 file system in partition #7 of SCSI1 (0,0,0) (sda).
Do you want to return to the partitioning menu?

please answer No)! /dev/sda8 is going to be our data partition (i.e., our NFS share).

After the basic installation make sure that you give server1 and server2 static IP addresses (server1: 192.168.0.172, server2: 192.168.0.173), as described at the beginning of http://www.howtoforge.com/perfect_setup_debian_sarge_p3).

Afterwards, you should check /etc/fstab on both systems. Mine looks like this on both systems:

# /etc/fstab: static file system information.
#
# proc /proc proc defaults 0 0 /dev/sda5 / ext3 defaults,errors=remount-ro 0 1 /dev/sda1 /boot ext3 defaults 0 2 /dev/sda6 none swap sw 0 0 /dev/hdc /media/cdrom0 iso9660 ro,user,noauto 0 0 /dev/fd0 /media/floppy0 auto rw,user,noauto 0 0

If you find that yours looks like this, for example:

# /etc/fstab: static file system information.
#
# proc /proc proc defaults 0 0 /dev/hda5 / ext3 defaults,errors=remount-ro 0 1 /dev/hda1 /boot ext3 defaults 0 2 /dev/hda6 none swap sw 0 0 /dev/hdc /media/cdrom0 iso9660 ro,user,noauto 0 0 /dev/fd0 /media/floppy0 auto rw,user,noauto 0 0

then please make sure you use /dev/hda instead of /dev/sda in the following configuration files. Also make sure that /dev/sda7 (or /dev/hda7) and /dev/sda8 (or /dev/hda8...) are not listed in /etc/fstab!

3 Synchronize System Time

It's important that both server1 and server2 have the same system time. Therefore we install an NTP client on both:

server1/server2:

apt-get install ntp ntpdate

Afterwards you can check that both have the same time by running

server1/server2:

date

Share this page:

19 Comment(s)

Add comment

Comments

From: Anonymous at: 2008-11-03 05:57:37
From: Clearjet at: 2009-06-29 15:53:39

The text says:


Also, make sure /dev/sda7 as well as /dev/sda8 are identical in size


But the illustration indicates:


/dev/sda7 -- 150 MB unmounted
/dev/sda8 -- 26 GB unmounted


So which is it?


Thanks

From: Anonymous at: 2009-07-03 11:50:05

Means same size on BOTH server in cluster.

From: gryger at: 2010-10-04 22:13:07

And here: http://docs.homelinux.org another well explained tutorial about DRBD and NFS on Debian.

From: Anonymous at: 2014-03-05 19:18:07

This is somewhat of the set-up that I have been looking for, however when joining this “Highly Available NFS Server or a Balanced MySQL Cluster” with a “Loadbalanced High-Availability Web Server Apache Cluster”, my concerns are the IP's...


The tutorial for both “Loadbalanced High-Availability MySQL Cluster and Loadbalanced High-Availability Web Server Apache Cluster” utilize the same IP addresses…


Within this tutorial it’s mentioned “Virtual IP address that represents the NFS cluster to the outside and also a NFS client IP address...”


I am looking to join two of the clusters to make a highly available stable web hosting cluster with utilizing either NFS or MySQL for the back-end…


Which IP’s should be used for each node?

From: Anonymous at: 2006-07-13 21:56:56

this may be pretty obvious, but when you install the
kernel-headers  package, make sure you're using the version which
match your running kernel (for example kernel-headers-2.6.8-2-686-smp ).

 

From: at: 2007-01-11 05:23:25

Yes, this is VERY IMPORTANT.


 When installing your kernel headers, simply do this:


apt-get install kernel-headers-`uname -r` drbd0.7-module-source drbd0.7-utils


 


The `uname -r` will automatically insert your proper kernel version into the command.


Try running the command uname -r once, by itself, to see.

From: Jason Priebe at: 2009-04-08 01:17:40

We considered the DRBD approach as well when we looked to replace our NetApp cluster with a linux-based solution.  We settled on a slightly different approach (using RHEL and Cluster Suite). I welcome you to read my blog post about it.

From: Anonymous at: 2006-03-26 22:06:50


with recent drbd utils (0.7.17), I had to do

drbdsetup /dev/drbd0 primary --do-what-I-say


From: Anonymous at: 2006-03-26 22:09:26


Also before doing mount -t ext3 /dev/drbd0 /data, you should of course create filesystem there

mkfs.ext3 /dev/drbd0

I suggest making XFS filesystem.

From: Anonymous at: 2009-08-04 12:48:43

On version 8.0.14, I have to do :


drbdsetup /dev/drbd0 primary -o

From: Anonymous at: 2009-11-07 15:17:35

Using drbd8-ultils you should use:

 drbdadm -- --overwrite-data-of-peer  primary all

From: Anonymous at: 2009-11-07 15:18:27

Using drbd8-ultils you should use:

 drbdadm -- --overwrite-data-of-peer  primary all

From: starzinger at: 2006-03-10 10:24:13


To enable automatic failback from server2 to server1, you need to put in the following:

/etc/heartbeat/ha.cf:

auto_failback on

From: Anonymous at: 2006-03-07 16:30:56

If I want the data to be available to the NFS machines themselves, do you recommend mounting the virtual IP on them?

From: Anonymous at: 2006-03-09 18:46:27


Thanks for your info, pretty interesting.

Just two questions:

- Why ext3 is your choice instead of reiserfs?

- Why are you using ip-alias instead of iproute2?

Thanks in advance.

From: Anonymous at: 2006-03-13 10:48:37


I've thought about doing this before, but using iSCSI and the built in /dev/md aka software RAID to link the devices together as a mirrored device. Since iSCSI is supposedly a more open standard and can be used with multiple operating systems it'll be easier to implement on non-Linux systems as well.

From: at: 2011-01-19 12:58:53

I think that is the only way,


why we are trying to use nfs then?


In drdb just one node is active, you cant do changes on the pasive one.


 

From: arkarwmh at: 2015-02-09 10:46:19

But how do i "Virtual Ip" over the 2 servers?