Distributed Replicated Storage Across Four Storage Nodes With GlusterFS 3.2.x On CentOS 6.3 - Page 2

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Sun, 2013-01-06 18:44. ::

4 Setting Up The GlusterFS Client

client1.example.com:

On the client, we can install the GlusterFS client as follows:

yum install glusterfs-client

Then we create the following directory:

mkdir /mnt/glusterfs

That's it! Now we can mount the GlusterFS filesystem to /mnt/glusterfs with the following command:

mount.glusterfs server1.example.com:/testvol /mnt/glusterfs

(Instead of server1.example.com you can as well use server2.example.com or server3.example.com or server4.example.com in the above command!)

You should now see the new share in the outputs of...

mount

[root@client1 ~]# mount
/dev/mapper/vg_client1-LogVol00 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
server1.example.com:/testvol on /mnt/glusterfs type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)
[root@client1 ~]#

... and...

df -h

[root@client1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_client1-LogVol00
                      9.7G  1.7G  7.5G  19% /
tmpfs                 499M     0  499M   0% /dev/shm
/dev/sda1             504M   39M  440M   9% /boot
server1.example.com:/testvol
                       58G  2.1G   53G  4% /mnt/glusterfs
[root@client1  ~]#

Instead of mounting the GlusterFS share manually on the client, you could modify /etc/fstab so that the share gets mounted automatically when the client boots.

Open /etc/fstab and append the following line:

vi /etc/fstab

[...]
server1.example.com:/testvol /mnt/glusterfs glusterfs defaults,_netdev 0 0

(Again, instead of server1.example.com you can as well use server2.example.com or server3.example.com or server4.example.com!)

To test if your modified /etc/fstab is working, reboot the client:

reboot

After the reboot, you should find the share in the outputs of...

df -h

... and...

mount

 

5 Testing

Now let's create some test files on the GlusterFS share:

client1.example.com:

touch /mnt/glusterfs/test1
touch /mnt/glusterfs/test2
touch /mnt/glusterfs/test3
touch /mnt/glusterfs/test4
touch /mnt/glusterfs/test5
touch /mnt/glusterfs/test6

Now let's check the /data directory on server1.example.com, server2.example.com, server3.example.com, and server4.example.com. You will notice that replication1 as well as replication2 hold only a part of the files/directories that make up the GlusterFS share on the client, but the nodes that make up replication1 (server1 and server2) or replication2 (server3 and server4) contain the same files (mirroring):

server1.example.com:

ls -l /data

[root@server1 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test1
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test2
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test4
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test5
[root@server1 ~]#

server2.example.com:

ls -l /data

[root@server2 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test1
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test2
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test4
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test5
[root@server2 ~]#

server3.example.com:

ls -l /data

[root@server3 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test3
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test6
[root@server3 ~]#

server4.example.com:

ls -l /data

[root@server4 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test3
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test6
[root@server4 ~]#

Now we shut down server1.example.com and server4.example.com and add/delete some files on the GlusterFS share on client1.example.com.

server1.example.com/server4.example.com:

shutdown -h now

client1.example.com:

rm -f /mnt/glusterfs/test5
rm -f /mnt/glusterfs/test6

The changes should be visible in the /data directory on server2.example.com and server3.example.com:

server2.example.com:

ls -l /data

[root@server2 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test1
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test2
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test4
[root@server2 ~]#

server3.example.com:

ls -l /data

[root@server3 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test3
[root@server3 ~]#

Let's boot server1.example.com and server4.example.com again and take a look at the /data directory:

server1.example.com:

ls -l /data

[root@server1 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test1
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test2
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test4
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test5
[root@server1 ~]#

server4.example.com:

ls -l /data

[root@server4 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test3
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test6
[root@server4 ~]#

As you see, server1.example.com and server4.example.com haven't noticed the changes that happened while they were down. This is easy to fix, all we need to do is invoke a read command on the GlusterFS share on client1.example.com, e.g.:

client1.example.com:

ls -l /mnt/glusterfs/

[root@client1 ~]# ls -l /mnt/glusterfs/
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test1
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test2
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test3
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test4
[root@client1 ~]#

Now take a look at the /data directory on server1.example.com and server4.example.com again, and you should see that the changes have been replicated to these nodes:

server1.example.com:

ls -l /data

[root@server1 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test1
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test2
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test4
[root@server1 ~]#

server4.example.com:

ls -l /data

[root@server4 ~]# ls -l /data
total 0
-rw-r--r-- 1 root root 0 2012-12-17 15:49 test3
[root@server4 ~]#

 

6 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.