High-Availability Storage Cluster With GlusterFS On Ubuntu

1. Introduction

Original article: http://blogama.org

In this tutorial I will show you how to install GlusterFS in a scalable way to create a storage cluster, starting with 2 servers on Ubuntu 8.04 LTS server. Files will be replicated and splitted accross all servers which is some sort of RAID 10 (raid 1 with < 4 servers). With 4 servers that have each 100GB hard drive, total storage will be 200GB and if one server fails, the data will still be intact and files on the failed server will be replicated on another working server.

GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86-64 server with SATA-II RAID and Infiniband HBA.

2. Installation

First you need to install some software:

sudo su

apt-get install sshfs build-essential flex bison byacc vim wget

Now we need to install fuse from source:

cd /root/
wget http://europe.gluster.org/glusterfs/fuse/fuse-2.7.4glfs11.tar.gz
tar -zxvf fuse-2.7.4glfs11.tar.gz
cd /root/fuse-2.7.4glfs11

Next we compile fuse:

./configure
make && make install

Next we will install GlusterFS:

cd /root/
wget http://ftp.gluster.com/pub/gluster/glusterfs/2.0/LATEST/glusterfs-2.0.0rc2.tar.gz
tar -zxvf glusterfs-2.0.0rc2.tar.gz
cd /root/glusterfs-2.0.0rc2/

Take a minute break and compile:

./configure
make && make install

For some reasons, libraries are going in the wrong directory so we need to (if someone has a clean fix to this please post it!):

cp /usr/local/lib/* -R /usr/lib/

Next we create some folders that will be used later on:

mkdir /mnt/glusterfs
mkdir /data/
mkdir /data/export
mkdir /data/export-ns
mkdir /etc/glusterfs/

3. Servers configuration

Before you go further, you need to know that GlusterFS works in a client/server way. What we will do is to make our servers both client and server for GlusterFS.

Lets start with the server configuration file ON ALL SERVERS:

vi /etc/glusterfs/glusterfs-server.vol

and make it look like this:

# file: /etc/glusterfs/glusterfs-server.vol
 
volume posix
  type storage/posix
  option directory /data/export
end-volume
 
volume locks
  type features/locks
  subvolumes posix
end-volume
 
volume brick
  type performance/io-threads
  option thread-count 8
  subvolumes locks
end-volume
 
volume posix-ns
  type storage/posix
  option directory /data/export-ns
end-volume
 
volume locks-ns
  type features/locks
  subvolumes posix-ns
end-volume
 
volume brick-ns
  type performance/io-threads
  option thread-count 8
  subvolumes locks-ns
end-volume
 
volume server
  type protocol/server
  option transport-type tcp
  option auth.addr.brick.allow *
  option auth.addr.brick-ns.allow *
  subvolumes brick brick-ns
end-volume

Now do:

glusterfsd -f /etc/glusterfs/glusterfs-server.vol

to start the server daemon.