How to install and setup DRBD on CentOS
This tutorial explains how to install and setup DRBD for your server. Before I start, let me explain what actually DRBD represents and what it is used for. DRBD stands for Distributed Replicated Block Device, a software-based, shared-nothing, replicated storage solution for mirroring the content of block devices such as hard disks, partitions, logical volumes etc. The main purpose of DRBD is to provide an alternative solution to encounter scenarios of High Availability such as FailOver or Disaster Recovery. You can view the fundamental detail on it's website here .
1. Preliminary note
For this tutorial, I am using CentOS 6.4 in the 32bit version. The example end result will show how to create files or folders inside a partition on Server A and successfully show all the content on another server (Server B) using DRBD .
Below is the sketch of the server architecture :-
+-----------------------------------+ +-----------------------------+
| [ DRBD Server A ] | 192.168.43.101 | 192.168.43.102 | [ DRBD Server B ] |
| OEL641 +----------------------+-----------------------+ OEL642 |
| folderA | | folderB |
+-----------------------------------+ +-----------------------------+
This tutorial is compatible with any version of CentOS/Redhat/Oracle Linux 6 . It's necessary that the server you'd like to install DRBD on has a free block-device. This example shows how to configure and use the block-device /dev/sdb1.
2. Installation Phase
Before we start, we need to update the system and install the required packages for the DRBD dependencies. Below are the commands that need to be run and the list of packages that get updated or installed. For this cases, we are using the yum command to get the latest packages from the public CentOS repository.
yum -y update
yum -y install gcc make automake autoconf libxslt libxslt-devel flex rpm-build kernel-devel
Setting up Install Process
Package gcc-4.4.7-3.el6.i686 already installed and latest version
Package 1:make-3.81-20.el6.i686 already installed and latest version
Package automake-1.11.1-4.el6.noarch already installed and latest version
Package autoconf-2.63-5.1.el6.noarch already installed and latest version
Package libxslt-1.1.26-2.0.2.el6_3.1.i686 already installed and latest version
Package libxslt-devel-1.1.26-2.0.2.el6_3.1.i686 already installed and latest version
Package flex-2.5.35-8.el6.i686 already installed and latest version
Package rpm-build-4.8.0-32.el6.i686 already installed and latest version
Package kernel-devel-2.6.32-358.el6.i686 already installed and latest version
We have to do the above step on both servers.
Once the dependency package installation has finished on both servers, we will start to install the DRBD packages. The packages can be downloaded from this website.
Below are the steps.
First I create a folder under /root/ path with name BUILD, BUILDROOT etc. This path are needed to store the rpm file after we've build the DRBD packages.
mkdir -p /root/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
Download the DRBD packages.
wget http://oss.linbit.com/drbd/drbd-utils-latest.tar.gz http://oss.linbit.com/drbd/8.4/drbd-8.4.7-1.tar.gz
Extract package drbd-8.4.7-1.tar.gz and drbd-utils-latest.tar.gz files.
tar -zxvf drbd-8.4.7-1.tar.gz
tar -zxvf drbd-utils-latest.tar.gz
Compile DRBD package.
cd drbd-8.4.7-1
drbd-8.4.7-1]# make km-rpm
...
...
You have now:
/root/rpmbuild/RPMS/i686/drbd-km-debuginfo-8.4.7-1.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-km-2.6.32_358.el6.i686-8.4.7-1.i686.rpm
Compile and configure package.
cd ..
cd drbd-utils-8.9.6
./configure
make rpm
...
...
You have now:
/root/rpmbuild/RPMS/i686/drbd-utils-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-debuginfo-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-heartbeat-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-xen-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-km-debuginfo-8.4.7-1.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-pacemaker-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-bash-completion-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-km-2.6.32_358.el6.i686-8.4.7-1.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-8.9.6-1.el6.i686.rpm
/root/rpmbuild/RPMS/i686/drbd-udev-8.9.6-1.el6.i686.rpm
Go to /root/rpmbuild/RPMS/i686. The packages of rpm from compilation will exists inside there and install related rpm packages.
cd /root/rpmbuild/RPMS/i686
rpm -Uvh drbd-xen* drbd-udev* drbd-pacemaker* drbd-bash-completion* drbd-utils-*.rpm drbd-km-*.rpm drbd-8*
Preparing... ########################################### [100%]
1:drbd-utils ########################################### [ 13%]
2:drbd-xen ########################################### [ 25%]
3:drbd-udev ########################################### [ 38%]
4:drbd-pacemaker ########################################### [ 50%]
5:drbd-bash-completion ########################################### [ 63%]
6:drbd ########################################### [ 75%]
7:drbd-km-2.6.32_358.el6.########################################### [ 88%]
8:drbd-km-debuginfo ########################################### [100%]
Once done on the first server, do the same steps on server OEL642.
3. Verification phase
Now we are half way from completion. Let's focus on the OEL641 configuration side. First, verify your server hostname, this is important because the configuration of DRBD will require the same name as its hostname. Below are the steps:
uname -n
OEL641
Then update the DRBD module
modprobe drbd
lsmod |grep drbd
drbd 341783 0
libcrc32c 841 1 drbd
4. That's all, now we will create a new disk partition to act as DRDB disk. Here I have a new disk /dev/sdb . I'll create a partition /dev/sdb1 and use it as DRBD disk.
fdisk -l
Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
fdisk /dev/sdb
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130):
Using default value 130
Command (m for help): p
Device Boot Start End Blocks Id System
/dev/sdb1 1 130 1044193+ 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
fdisk -l
Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x89770d51
Device Boot Start End Blocks Id System
/dev/sdb1 1 130 1044193+ 83 Linux
Once done, repeat the steps on OEL642
5. DRBD Configuration
Now we've coming to final step which is the configuration of DRBD. Basically, we need to set which partition disk and server should act under the DRBD module. To do that, we will create a respond file as below and copy over to OEL642 so that both servers have same configuration.
cd /etc/drbd.d/
vi s1.res
resource s1 {
on OEL641 {
device /dev/drbd1;
disk /dev/sdb1;
address 192.168.43.101:7799;
meta-disk internal;
}
on OEL642 {
device /dev/drbd1;
disk /dev/sdb1;
address 192.168.43.102:7799;
meta-disk internal;
}
}
Copy the file with scp to the second server:
scp /etc/drbd.d/s1.res root@OEL642:/etc/drbd.d/s1.res
Once done, we need to initialize the partition on both servers. Below are the steps:
drbdadm create-md s1
initializing activity log
NOT initializing bitmap
Writing meta data...
New drbd meta data block successfully created.
success
s1 is resource name that we defined in the s1.res file above.
Please login to server OEL642 and do the same steps from above:
drbdadm create-md s1
initializing activity log
NOT initializing bitmap
Writing meta data...
New drbd meta data block successfully created.
success
Now go back at server OEL641 and we can start the DRBD service. Below are the steps :-
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
[root@OEL641 ~]# /etc/init.d/drbd start
Starting DRBD resources: [
create res: s1
prepare disk: s1
adjust disk: s1
adjust net: s1
]
..........
***************************************************************
DRBD's startup script waits for the peer node(s) to appear.
- If this node was already a degraded cluster before the
reboot, the timeout is 0 seconds. [degr-wfc-timeout]
- If the peer was available before the reboot, the timeout
is 0 seconds. [wfc-timeout]
(These values are for resource 's1'; 0 sec -> wait forever)
To abort waiting enter 'yes' [ 26]:
.
Notice in above that the DRBD service did start not yet. This is due to server OEL641 is looking for the DRBD service on OEL642.
Therefore during this time, please login into OEL642 and start the DRBD service in the same way you did om OEL641. You'll notice that once DRBD service at OEL642 is started, DRBD service at OEL641 will stop waiting and started also.
Once done, DRBD service status will show like below on both server:
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
1:s1 Connected Secondary/Secondary Inconsistent/Inconsistent C
Now do below steps on server OEL641 only. This is to initialize who will be the initial primary server.
drbdadm -- --overwrite-data-of-peer primary s1
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
... sync'ed: 0.8% (1037872/1044124)K
1:s1 SyncSource Primary/Secondary UpToDate/Inconsistent C
Then check on OEL642:
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
... sync'ed: 12.2% (921244/1044124)K
1:s1 SyncTarget Secondary/Primary Inconsistent/UpToDate C
You'll notice in above status report that now server OEL641 has been verified as the Primary and OEL642 as the Secondary server.
This means that now whatever files or folder we've create under DRBD partition on server OEL641 will be synced over to DRBD partition at OEL642.
6. DRBD Testing phase
Now, we've come to the final part which is testing of the DRBD service to ensure it meets the objective.
First, let's mount the DRBD partition.
Do the below steps once on the primary server OEL641 ONLY!
mkfs.ext3 /dev/drbd1
...
...
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This formats the partition with the ext3 filesystem.
Now create a folder and mount the DRBD partition to it.
mkdir /folderA
mount /dev/drbd1 /folderA
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/drbd1 1004M 18M 936M 2% /folderA
cd /folderA/
ls -h
lost+found
As shown above, the new folder called folderA has successfully been created.
Now let's create a file with a size of 30Mb inside the folder:
dd if=/dev/zero of=/folderA/testfile bs=1M count=30
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 0.178448 s, 176 MB/s
ls -lh
total 31M
drwx------. 2 root root 16K Aug 21 08:55 lost+found
-rw-r--r--. 1 root root 30M Aug 21 09:09 testfile
These have been the prerequisites, let's start to test the process.
As the final result, once we mount the DRBD partition on server OEL642, the test file should exists inside the folder.
Now let's unmount the DRBD partition in OEL641 and make it as secondary state. Secondary state means that you'll not able to mount the DRBD partition into Operating System to be use.
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
1:s1 Connected Primary/Secondary UpToDate/UpToDate C /folderA ext3
umount /folderA
drbdadm secondary s1
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
1:s1 Connected Secondary/Secondary UpToDate/UpToDate C
Now both DRBD servers are in secondary state. Let's go to server OEL642 and change it from Secondary to Primary state.
First create an empty folder and mount to DRBD partition on OEL642:
mkdir /folderB
cd /folderB
ls -lh
total 0
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
1:s1 Connected Secondary/Secondary UpToDate/UpToDate C
drbdadm primary s1
/etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.4.7-1 (api:1/proto:86-101)
GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by root@OEL641, 2016-08-20 01:23:44
m:res cs ro ds p mounted fstype
1:s1 Connected Primary/Secondary UpToDate/UpToDate C
mount /dev/drbd1 /folderB
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_oel641-lv_root
96G 3.5G 88G 4% /
tmpfs 640M 80K 640M 1% /dev/shm
/dev/sda1 485M 32M 428M 7% /boot
/dev/drbd1 1004M 48M 906M 5% /folderB
ls -lh /folderB/
total 31M
drwx------. 2 root root 16K Aug 21 08:55 lost+found
-rw-r--r--. 1 root root 30M Aug 21 09:09 testfile
Once you've successfully mounted the DRBD partition to your new folder, you'll notice that the file that you've created in server OEL641 before automatically existed!
DONE! You've successfully setup and configure DRBD on CentOS!
