How to install GlusterFS with a replicated volume over 2 nodes on Ubuntu 14.04

Version 1.0
Author: Srijan Kishore
Last edited: 16/July/2014

In this tutorial I will explain GlusterFS configuration in Ubuntu 14.04. GlusterFS is an open source distributed file system which provides easy replication over multiple storage nodes. Gluster File System is a distributed filesystem allowing you to create a single volume of storage which spans multiple disks, multiple machines and even multiple data centres.

I do not issue any guarantee that this will work for you!

1 Preliminary Note

In my case I have two Ubuntu 14.04 servers, with details as:

IP 192.168.0.100, hostname server1.example.com

IP 192.168.0.101, hostname server2.example.com

2 GlusterFS server installation

Gluster File System is a distributed files system allowing you to create a single volume of storage which spans multiple disks, multiple machines and even multiple data centres.

Install the required packages using apt-get on both Ubuntu machines. If you have more than two servers, perform this command on all of the servers required for the volume.

apt-get install glusterfs-server

After successful installation, open the terminal and type following command and check if the installation was successful:

 glusterfs --version

[email protected]:~#  glusterfs --version
glusterfs 3.4.2 built on Jan 14 2014 18:05:35
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
[email protected]:~#

[email protected]:~#  glusterfs --version
glusterfs 3.4.2 built on Jan 14 2014 18:05:35
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
[email protected]:~#

Now it is mandatory that both machines must listen to each other with their hostname, so I will update both Ubuntu 14.04 machines with the entries in /etc/hosts:

vi /etc/hosts

192.168.0.100	server1.example.com	gluster1
192.168.0.101	server2.example.com	gluster2

Now in both machines run the command  gluster peer probe I will run this on first Ubuntu 14.04 (server1.example.com) machine as follows:

gluster peer probe gluster1

[email protected]:~# gluster peer probe gluster1
peer probe: success: on localhost not needed

further

gluster peer probe gluster2

[email protected]:~# gluster peer probe gluster2
peer probe: success

Now I will check the status for peer as:

gluster peer status

[email protected]:~# gluster peer status
Number of Peers: 1
Hostname: gluster2
Port: 24007
Uuid: 5c5a045c-34b9-44ac-b5c0-8acb461d8523
State: Peer in Cluster (Connected)
[email protected]:~#

Again same thing I will repeat on second Ubuntu 14.04 (server2.example.com) machine:

 gluster peer probe gluster1

[email protected] ~ # gluster peer probe gluster1
peer probe: success

Next:

gluster peer probe gluster2

[email protected] ~ # gluster peer probe gluster2
peer probe: success: on localhost not needed

& check the peer status as follows:

gluster peer status

[email protected] ~ # gluster peer status
Number of Peers: 1

Hostname: gluster1
Port: 24007
Uuid: 8d865314-af12-4950-a784-6a5308ec501b
State: Peer in Cluster (Connected)
[email protected] ~ #

Now I will create a common folder on both Ubuntu 14.04 machines viz /mnt/gluster

mkdir -p /mnt/gluster

If you wish you can use any other mount points on both machines.

Now we need to create the volume where the data will reside. The volume will be called datapoint. Now run on any machine:

gluster  volume create datapoint replica 2 transport tcp  gluster1:/mnt/gluster  gluster2:/mnt/gluster force

[email protected]:~# gluster  volume create datapoint replica 2 transport tcp  gluster1:/mnt/gluster  gluster2:/mnt/gluster
volume create: datapoint: success: please start the volume to access data
[email protected]:~#

Now we need to start the volume:

gluster volume start datapoint

[email protected]:~# gluster volume start datapoint
volume start: datapoint: success
[email protected]:~#

Running either of the below commands should indicate that GlusterFS is up and running. The ps command should show the command running with both servers in the argument. netstat should show a connection between both nodes.

ps aux | grep gluster

[email protected]:~# ps aux | grep gluster
root      2041  0.0  0.8 391892 16252 ?        Ssl  11:49   0:00 /usr/sbin/glusterd -p /var/run/glusterd.pid
root      2865  0.0  1.0 451692 19464 ?        Ssl  14:06   0:00 /usr/sbin/glusterfsd -s gluster1 --volfile-id datapoint.gluster1.mnt-gluster -p /var/lib/glusterd/vols/datapoint/run/gluster1-mnt-gluster.pid -S /var/run/d317967a0e3119238993e1580556da73.socket --brick-name /mnt/gluster -l /var/log/glusterfs/bricks/mnt-gluster.log --xlator-option *-posix.glusterd-uuid=8d865314-af12-4950-a784-6a5308ec501b --brick-port 49152 --xlator-option datapoint-server.listen-port=49152
root      2875  0.0  2.8 277732 53404 ?        Ssl  14:06   0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /var/run/d3557e241e521ea123bcdfb9ed54e30f.socket
root      2882  0.0  1.2 295436 23492 ?        Ssl  14:06   0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/lib/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/f06a6deb150e1c5c0e607ec357f085f4.socket --xlator-option *replicate*.node-uuid=8d865314-af12-4950-a784-6a5308ec501b
root      2900  0.0  0.0  11744   924 pts/0    S+   14:09   0:00 grep --color=auto gluster
[email protected]:~#

netstat -tap | grep glusterfsd

[email protected]:~# netstat -tap | grep glusterfsd
tcp        0      0 *:49152                 *:*                     LISTEN      2865/glusterfsd 
tcp        0      0 server1.example.c:49152 server2.example:1020 ESTABLISHED 2865/glusterfsd 
tcp        0      0 server1.example.co:1019 server1.example.c:24007 ESTABLISHED 2865/glusterfsd 
tcp        0      0 server1.example.c:49152 server1.example.co:1023 ESTABLISHED 2865/glusterfsd 
tcp        0      0 server1.example.c:49152 server2.example:1014 ESTABLISHED 2865/glusterfsd 
tcp        0      0 server1.example.c:49152 server1.example.co:1022 ESTABLISHED 2865/glusterfsd 
[email protected]:~#

 

As a final test, to make sure the volume is available, run gluster volume info. As shown below:

gluster volume info

[email protected]:~# gluster volume info
 
Volume Name: datapoint
Type: Replicate
Volume ID: 3fd7bcea-3ee5-41b4-9336-880a5c1527b7
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: gluster1:/mnt/gluster
Brick2: gluster2:/mnt/gluster
[email protected]:~#

 

It shows that two bricks are fully functional. Now we have a GlusterFS volume which will maintain replication across two nodes. In the same way you can use N numbers of brick as nodes.

Congrats you have done with the GlusterFS configuration in ubuntu14.04 :)

Suggested articles

7 Comment(s)

Comments