How to install GlusterFS with a replicated volume over 2 nodes on Ubuntu 14.04
Version 1.0
Author: Srijan Kishore
Last edited: 16/July/2014
In this tutorial I will explain GlusterFS configuration in Ubuntu 14.04. GlusterFS is an open source distributed file system which provides easy replication over multiple storage nodes. Gluster File System is a distributed filesystem allowing you to create a single volume of storage which spans multiple disks, multiple machines and even multiple data centres.
I do not issue any guarantee that this will work for you!
1 Preliminary Note
In my case I have two Ubuntu 14.04 servers, with details as:
IP 192.168.0.100, hostname server1.example.com
IP 192.168.0.101, hostname server2.example.com
2 GlusterFS server installation
Gluster File System is a distributed files system allowing you to create a single volume of storage which spans multiple disks, multiple machines and even multiple data centres.
Install the required packages using apt-get on both Ubuntu machines. If you have more than two servers, perform this command on all of the servers required for the volume.
apt-get install glusterfs-server
After successful installation, open the terminal and type following command and check if the installation was successful:
glusterfs --versionroot@server1:~# glusterfs --version
glusterfs 3.4.2 built on Jan 14 2014 18:05:35
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
root@server1:~#
root@server2:~# glusterfs --version
glusterfs 3.4.2 built on Jan 14 2014 18:05:35
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
root@server2:~#
Now it is mandatory that both machines must listen to each other with their hostname, so I will update both Ubuntu 14.04 machines with the entries in /etc/hosts:
vi /etc/hosts
192.168.0.100 server1.example.com gluster1
192.168.0.101 server2.example.com gluster2
Now in both machines run the command gluster peer probe I will run this on first Ubuntu 14.04 (server1.example.com) machine as follows:
gluster peer probe gluster1
root@server1:~# gluster peer probe gluster1 peer probe: success: on localhost not needed
further
gluster peer probe gluster2
root@server1:~# gluster peer probe gluster2
peer probe: success
Now I will check the status for peer as:
gluster peer status
root@server1:~# gluster peer status
Number of Peers: 1
Hostname: gluster2
Port: 24007
Uuid: 5c5a045c-34b9-44ac-b5c0-8acb461d8523
State: Peer in Cluster (Connected)
root@server1:~#
Again same thing I will repeat on second Ubuntu 14.04 (server2.example.com) machine:
gluster peer probe gluster1
root@server2 ~ # gluster peer probe gluster1 peer probe: success
Next:
gluster peer probe gluster2
root@vboxtest ~ # gluster peer probe gluster2 peer probe: success: on localhost not needed
& check the peer status as follows:
gluster peer status
root@server2 ~ # gluster peer status
Number of Peers: 1
Hostname: gluster1
Port: 24007
Uuid: 8d865314-af12-4950-a784-6a5308ec501b
State: Peer in Cluster (Connected)
root@vboxtest ~ #
Now I will create a common folder on both Ubuntu 14.04 machines viz /mnt/gluster
mkdir -p /mnt/gluster
If you wish you can use any other mount points on both machines.
Now we need to create the volume where the data will reside. The volume will be called datapoint. Now run on any machine:
gluster volume create datapoint replica 2 transport tcp gluster1:/mnt/gluster gluster2:/mnt/gluster force
root@server1:~# gluster volume create datapoint replica 2 transport tcp gluster1:/mnt/gluster gluster2:/mnt/gluster volume create: datapoint: success: please start the volume to access data root@server1:~#
Now we need to start the volume:
gluster volume start datapoint
root@server1:~# gluster volume start datapoint volume start: datapoint: success root@server1:~#
Running either of the below commands should indicate that GlusterFS is up and running. The ps command should show the command running with both servers in the argument. netstat should show a connection between both nodes.
ps aux | grep gluster
root@server1:~# ps aux | grep gluster root 2041 0.0 0.8 391892 16252 ? Ssl 11:49 0:00 /usr/sbin/glusterd -p /var/run/glusterd.pid root 2865 0.0 1.0 451692 19464 ? Ssl 14:06 0:00 /usr/sbin/glusterfsd -s gluster1 --volfile-id datapoint.gluster1.mnt-gluster -p /var/lib/glusterd/vols/datapoint/run/gluster1-mnt-gluster.pid -S /var/run/d317967a0e3119238993e1580556da73.socket --brick-name /mnt/gluster -l /var/log/glusterfs/bricks/mnt-gluster.log --xlator-option *-posix.glusterd-uuid=8d865314-af12-4950-a784-6a5308ec501b --brick-port 49152 --xlator-option datapoint-server.listen-port=49152 root 2875 0.0 2.8 277732 53404 ? Ssl 14:06 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /var/run/d3557e241e521ea123bcdfb9ed54e30f.socket root 2882 0.0 1.2 295436 23492 ? Ssl 14:06 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/lib/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/f06a6deb150e1c5c0e607ec357f085f4.socket --xlator-option *replicate*.node-uuid=8d865314-af12-4950-a784-6a5308ec501b root 2900 0.0 0.0 11744 924 pts/0 S+ 14:09 0:00 grep --color=auto gluster root@server1:~#
netstat -tap | grep glusterfsd
root@server1:~# netstat -tap | grep glusterfsd tcp 0 0 *:49152 *:* LISTEN 2865/glusterfsd tcp 0 0 server1.example.c:49152 server2.example:1020 ESTABLISHED 2865/glusterfsd tcp 0 0 server1.example.co:1019 server1.example.c:24007 ESTABLISHED 2865/glusterfsd tcp 0 0 server1.example.c:49152 server1.example.co:1023 ESTABLISHED 2865/glusterfsd tcp 0 0 server1.example.c:49152 server2.example:1014 ESTABLISHED 2865/glusterfsd tcp 0 0 server1.example.c:49152 server1.example.co:1022 ESTABLISHED 2865/glusterfsd root@server1:~#
As a final test, to make sure the volume is available, run gluster volume info. As shown below:
gluster volume info
root@server1:~# gluster volume info Volume Name: datapoint Type: Replicate Volume ID: 3fd7bcea-3ee5-41b4-9336-880a5c1527b7 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: gluster1:/mnt/gluster Brick2: gluster2:/mnt/gluster root@server1:~#
It shows that two bricks are fully functional. Now we have a GlusterFS volume which will maintain replication across two nodes. In the same way you can use N numbers of brick as nodes.
Congrats you have done with the GlusterFS configuration in ubuntu14.04 :)