Setting Up Network RAID1 With DRBD On Ubuntu 11.10 - Page 2

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Sun, 2011-10-30 20:36. ::

4 Install And Configure DRBD

server1/server2:

Now install DRBD on both nodes as follows:

apt-get install drbd8-utils

Load the DRBD kernel module:

modprobe drbd

To check if it is loaded, run:

lsmod | grep drbd

Output should be similar to this one:

root@server1:~# lsmod | grep drbd
drbd                  273002  0
lru_cache              14896  1 drbd
root@server1:~#

Now we back up the original /etc/drbd.conf file and create a new one on both nodes as follows:

cp /etc/drbd.conf /etc/drbd.conf_orig
cat /dev/null > /etc/drbd.conf
vi /etc/drbd.conf

global { usage-count no; }
common { syncer { rate 100M; } }
resource r0 {
        protocol C;
        startup {
                wfc-timeout  15;
                degr-wfc-timeout 60;
        }
        net {
                cram-hmac-alg sha1;
                shared-secret "secret";
        }
        on server1.example.com {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.0.100:7788;
                meta-disk internal;
        }
        on server2.example.com {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.0.101:7788;
                meta-disk internal;
        }
}

Make sure you use the correct node names in the file (instead of server1.example.com and server2.example.com) - please make sure you use the node names that the command

uname -n

shows on both nodes. Also make sure you fill in the correct IP addresses in the address lines and the correct disk in the disk lines (if you don't use /dev/sdb1).

Now we initialize the meta data storage. On both nodes run:

drbdadm create-md r0

root@server1:~# drbdadm create-md r0
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
root@server1:~#

Then start DRBD on both nodes:

/etc/init.d/drbd start

root@server1:~# /etc/init.d/drbd start
 * Starting DRBD resources [ d(r0) s(r0) n(r0) ] [ OK ]
root@server1:~#

The next step has to be carried out on server1 only:

server1:

Now make server1 the primary node:

drbdadm -- --overwrite-data-of-peer primary all

Afterwards, data will start to synchronize between server1 and server2.

server2:

Take a look at

cat /proc/drbd

to see the synchronization progress:

root@server2:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
    ns:0 nr:10166400 dw:10166400 dr:0 al:0 bm:620 lo:1 pe:7407 ua:0 ap:0 ep:1 wo:f oos:21288860
        [=====>..............] sync'ed: 32.4% (20788/30716)Mfinish: 0:03:53 speed: 91,180 (86,152) want: 102,400 K/sec
root@server2:~#

(You can run

watch cat /proc/drbd

to get an ongoing output of the process. To leave watch, press CTRL+C.)

Wait until the synchronization has finished - output should be as follows:

root@server2:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:31455260 dw:31455260 dr:0 al:0 bm:1909 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server2:~#

The snippet ro:Secondary/Primary tells you that this node is the secondary node.

server1:

On server1, the output of

cat /proc/drbd

is as follows (after the synchronization has finished):

root@server1:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
    ns:31455260 nr:0 dw:0 dr:31455924 al:0 bm:1920 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server1:~#

The snippet ro:Primary/Secondary tells you that this is the primary node.

Now that we have our new network RAID1 block device /dev/drbd0 (which consists of /dev/sdb1 from server1 and server2), let's create an ext4 filesystem on it and mount it to the /data directory. This has to be done only on server1!

mkfs.ext4 /dev/drbd0
mkdir /data
mount /dev/drbd0 /data

Afterwards you should see /dev/drbd0 in the outputs of...

mount

root@server1:~# mount
/dev/mapper/server1-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)
/dev/drbd0 on /data type ext4 (rw)
root@server1:~#

... and:

df -h

root@server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/server1-root
                       29G 1017M   27G   4% /
udev                  238M  4.0K  238M   1% /dev
tmpfs                  99M  228K   99M   1% /run
none                  5.0M  4.0K  5.0M   1% /run/lock
none                  247M     0  247M   0% /run/shm
/dev/sda1             228M   24M  193M  11% /boot
/dev/drbd0             30G  172M   28G   1% /data
root@server1:~#

 

5 Test

server1:

Now let's create some files or directories in the /data directory and check whether they get replicated to server2.

touch /data/test1.txt
touch /data/test2.txt

ls -l /data/

root@server1:~# ls -l /data/
total 16
drwx------ 2 root root 16384 2011-10-28 14:12 lost+found
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test1.txt
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test2.txt
root@server1:~#

Now let's unmount the /data directory on server1:

umount /data

Then assign the secondary role to server1:

drbdadm secondary r0

Now we go to server2, make it the primary node and check if we can see the files/directories we created on server1 in the /data directory on server2.

server2:

First we assign the primary role to server2:

drbdadm primary r0

Check the output of

cat /proc/drbd

... and you should see that server2 is the primary node now:

root@server2:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
    ns:0 nr:31691444 dw:31691444 dr:664 al:0 bm:1909 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server2:~#

Next we create the /data directory and mount /dev/drbd0 to it:

mkdir /data
mount /dev/drbd0 /data

Let's check the contents of the /data directory:

ls -l /data/

If everything went fine, it should contain the files/directories that we created on server1:

root@server2:~# ls -l /data/
total 16
drwx------ 2 root root 16384 2011-10-28 14:12 lost+found
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test1.txt
-rw-r--r-- 1 root root     0 2011-10-28 14:13 test2.txt
root@server2:~#

server1:

Now that we have switched roles, the output of

cat /proc/drbd

on server1 should show you that server1 has the secondary role:

root@server1:~# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: DA5A13F16DE6553FC7CE9B2
 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:31691444 nr:185568 dw:421752 dr:31457005 al:83 bm:1920 lo:1 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
root@server1:~#

 

6 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by SamTzu (registered user) on Mon, 2012-01-16 08:40.

Nice tutorial, thx.

What about authentication between these servers? I don't see how that was done.

Submitted by Ziemowit Pierzycki (not registered) on Thu, 2011-11-03 01:13.
The only problem I see with this HOWTO is that you're using a non-clustered file system like EXT4.  This won't work well.  While one can mount EXT4 on both nodes, only one will be able write.  I recommend something like GFS2 or OCFS2.