Setting Up Network RAID1 With DRBD On Ubuntu 11.10 - Page 2

4 Install And Configure DRBD

server1/server2:

Now install DRBD on both nodes as follows:

apt-get install drbd8-utils

Load the DRBD kernel module:

modprobe drbd

To check if it is loaded, run:

lsmod | grep drbd

Output should be similar to this one:

root@server1:~# lsmod | grep drbd
drbd                  273002  0
lru_cache              14896  1 drbd
root@server1:~#

Now we back up the original /etc/drbd.conf file and create a new one on both nodes as follows:

cp /etc/drbd.conf /etc/drbd.conf_orig
cat /dev/null > /etc/drbd.conf
vi /etc/drbd.conf

global { usage-count no; }
common { syncer { rate 100M; } }
resource r0 {
        protocol C;
        startup {
                wfc-timeout  15;
                degr-wfc-timeout 60;
        }
        net {
                cram-hmac-alg sha1;
                shared-secret "secret";
        }
        on server1.example.com {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.0.100:7788;
                meta-disk internal;
        }
        on server2.example.com {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.0.101:7788;
                meta-disk internal;
        }
}

Make sure you use the correct node names in the file (instead of server1.example.com and server2.example.com) - please make sure you use the node names that the command

uname -n

shows on both nodes. Also make sure you fill in the correct IP addresses in the address lines and the correct disk in the disk lines (if you don't use /dev/sdb1).

Now we initialize the meta data storage. On both nodes run:

drbdadm create-md r0

root@server1:~# drbdadm create-md r0
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
root@server1:~#

Then start DRBD on both nodes:

/etc/init.d/drbd start

root@server1:~# /etc/init.d/drbd start
 * Starting DRBD resources [ d(r0) s(r0) n(r0) ] [ OK ]
root@server1:~#

The next step has to be carried out on server1 only:

server1:

Now make server1 the primary node:

drbdadm -- --overwrite-data-of-peer primary all

Afterwards, data will start to synchronize between server1 and server2.

server2:

Take a look at

cat /proc/drbd

to see the synchronization progress:

root@server2:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
    ns:0 nr:10166400 dw:10166400 dr:0 al:0 bm:620 lo:1 pe:7407 ua:0 ap:0 ep:1 wo:f oos:21288860
        [=====>..............] sync'ed: 32.4% (20788/30716)Mfinish: 0:03:53 speed: 91,180 (86,152) want: 102,400 K/sec
root@server2:~#

(You can run

watch cat /proc/drbd

to get an ongoing output of the process. To leave watch, press CTRL+C.)

Wait until the synchronization has finished - output should be as follows:

root@server2:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:31455260 dw:31455260 dr:0 al:0 bm:1909 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server2:~#

The snippet ro:Secondary/Primary tells you that this node is the secondary node.

server1:

On server1, the output of

cat /proc/drbd

is as follows (after the synchronization has finished):

root@server1:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
    ns:31455260 nr:0 dw:0 dr:31455924 al:0 bm:1920 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server1:~#

The snippet ro:Primary/Secondary tells you that this is the primary node.

Now that we have our new network RAID1 block device /dev/drbd0 (which consists of /dev/sdb1 from server1 and server2), let's create an ext4 filesystem on it and mount it to the /data directory. This has to be done only on server1!

mkfs.ext4 /dev/drbd0
mkdir /data
mount /dev/drbd0 /data

Afterwards you should see /dev/drbd0 in the outputs of...

mount

root@server1:~# mount
/dev/mapper/server1-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)
/dev/drbd0 on /data type ext4 (rw)
root@server1:~#

... and:

df -h

root@server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/server1-root
                       29G 1017M   27G   4% /
udev                  238M  4.0K  238M   1% /dev
tmpfs                  99M  228K   99M   1% /run
none                  5.0M  4.0K  5.0M   1% /run/lock
none                  247M     0  247M   0% /run/shm
/dev/sda1             228M   24M  193M  11% /boot
/dev/drbd0             30G  172M   28G   1% /data
root@server1:~#

 

5 Test

server1:

Now let's create some files or directories in the /data directory and check whether they get replicated to server2.

touch /data/test1.txt
touch /data/test2.txt

ls -l /data/

root@server1:~# ls -l /data/
total 16
drwx------ 2 root root 16384 2011-10-28 14:12 lost+found
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test1.txt
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test2.txt
root@server1:~#

Now let's unmount the /data directory on server1:

umount /data

Then assign the secondary role to server1:

drbdadm secondary r0

Now we go to server2, make it the primary node and check if we can see the files/directories we created on server1 in the /data directory on server2.

server2:

First we assign the primary role to server2:

drbdadm primary r0

Check the output of

cat /proc/drbd

... and you should see that server2 is the primary node now:

root@server2:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
    ns:0 nr:31691444 dw:31691444 dr:664 al:0 bm:1909 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server2:~#

Next we create the /data directory and mount /dev/drbd0 to it:

mkdir /data
mount /dev/drbd0 /data

Let's check the contents of the /data directory:

ls -l /data/

If everything went fine, it should contain the files/directories that we created on server1:

root@server2:~# ls -l /data/
total 16
drwx------ 2 root root 16384 2011-10-28 14:12 lost+found
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test1.txt
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test2.txt
root@server2:~#

server1:

Now that we have switched roles, the output of

cat /proc/drbd

on server1 should show you that server1 has the secondary role:

root@server1:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:31691444 nr:185568 dw:421752 dr:31457005 al:83 bm:1920 lo:1 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server1:~#

 

6 Links

Share this page:

2 Comment(s)

Add comment

Comments

From: Ziemowit Pierzycki at: 2011-11-03 00:13:27

The only problem I see with this HOWTO is that you're using a non-clustered file system like EXT4.  This won't work well.  While one can mount EXT4 on both nodes, only one will be able write.  I recommend something like GFS2 or OCFS2.

From: at: 2012-01-16 07:40:05

Nice tutorial, thx.

What about authentication between these servers? I don't see how that was done.