GlusterFS Installation on Ubuntu 22.04

Introduction

GlusterFS is an open-source, distributed file system that allows you to combine multiple storage servers into a single, high-performance storage cluster. It provides scalability, high availability as well as, fault tolerance, making it an excellent choice for managing large amounts of data across multiple hosts. This article will walk you through the step-by-step process of GlusterFS version 10.1 installation on three Ubuntu hosts, enabling you to create a robust and reliable distributed file system.

Installation Prerequisites

Before starting the installation process, ensure that you have the following:

• Three Ubuntu hosts (physical or virtual machines) running Ubuntu OS 22.04.
• Root access or sudo privileges on all three hosts.
• A network connection between the hosts.

We are going to demonstrate installation and configuration on the 3 virtual machines hosted on VMware ESXi hypervisor.

 

Virtual Machine Specs

Installation Procedure

Below are the step by step instructions to install and configure 3 node GlusterFS storage.

GlusterFS User Creation

Create GlusterFS service user account with sudo privileges on each host which will be used during installation procedure.

$ sudo useradd glusterfsuser
$ sudo passwd glusterfsuser

Add the below line in /etc/sudoers to alleviate glusterfsuser to sudo privileges without password prompt.

$ sudo visudo
…
glsfsuser  ALL=(ALL)  NOPASSWD:ALL
…

Configure Host Name and IP Addresses

Login to each host with sudo user glsfsuser as mentioned above assign unique hostnames to each Ubuntu machine to ensure proper communication within the GlusterFS cluster. Edit the `/etc/hostname` file on each host and set the desired hostname. Then, edit the `/etc/hosts` file to associate each hostname with its corresponding IP address.

If DNS server available in the environment configure DNS A record in DNS server and configure DNS resolution.  It is recommended to use DNS server in production environment. However, in the procedure we are using /etc/hosts files for name resolution.

$ sudo vi /etc/hosts
…
192.168.0.101  glsfshost01.computerologists.com  glsfshost01
192.168.0.102  glsfshost02.computerologists.com  glsfshost02
192.168.0.103  glsfshost03.computerologists.com  glsfshost03
…

Update System Packages

To begin, update the system packages to their latest versions by running the following command on each host.

$ sudo apt update && sudo apt upgrade -y

Install GlusterFS Software Packages

On each Ubuntu host, install GlusterFS by executing the following command. This command installs the GlusterFS server package on each host.

$ sudo apt install glusterfs-server -y

$ sudo gluster --version
glusterfs 10.1
Repository revision: git://git.gluster.org/glusterfs.git
Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.

Start and Enable GlusterFS Service

Start GlusterFS service and also enable the service at startup to make sure in case of reboot GlusterFS service starts at boot on each host.

$ sudo systemctl start glusterd.service
$ sudo systemctl enable glusterd.service
$ sudo systemctl status glusterd.service

Opening GlusterFS ports in Firewall

Run the below commands on each host to open GlusterFS ports in Firewall to make sure GlusterFS is accessible by peer nodes and clients.

$ sudo ufw allow 24007/tcp
$ sudo ufw allow 24008/tcp
$ sudo ufw allow 24009/tcp
$ sudo ufw allow 49152/tcp
$ sudo ufw allow 111/tcp
$ sudo ufw allow 38465-38467/tcp
$ sudo ufw allow 38468/tcp

Set Up GlusterFS Peers

On the first Ubuntu host glsfshost01, run the following command to add the other two hosts glsfshost02 and glsfshost03 to peer all three hosts in a single pool

$ sudo gluster peer probe  glsfshost02
$ sudo gluster peer probe  glsfshost03

Run below command on first host glsfshost01 to verify hosts peering status and pool status.

$ sudo gluster peer status
Number of Peers: 2
Hostname: glsfshost02
Uuid: 2b95c367-8e3c-428d-8e8b-21b2774ac717
State: Peer in Cluster (Connected)


Hostname: glsfshost03
Uuid: e989e7b0-15ae-4fef-9d92-0181610ae641
State: Peer in Cluster (Connected)

Create GlusterFS Volume

Now, to create a GlusterFS volume that spans across all three hosts. we ‘ll create logical volume on unallocated disk /dev/sdb on each host.

$ sudo lvm
lvm> pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
lvm> vgcreate glsfsvg01 /dev/sdb
Volume group "glsfsvg01" successfully created
lvm> lvcreate -l 100%FREE -n  vol01 glsfsvg01
Logical volume "vol01" created.
lvm> exit

Format Logical volume with file system and mount the same on all three hosts.

$ sudo mkfs.xfs /dev/glsfsvg01/vol01 
$ sudo mkdir /glsvol01
$ sudo mount /dev/dev/glsfsvg01/vol01 /glsvol01

To make this file system persistent mounting add the below line in /etc/fstab.

$ sudo vi /etc/fstab
…
/dev/glsfsvg01/vol01 /glsvol01 xfs defaults 0 0
…

Setup GlusterFS Replicated Volume

Create brick directories on all three hosts in the created and mounted volume in previous step to create a replicated GlusterFS volume across all three hosts.

$ sudo mkdir /glsvol01/brick01

Run the below command on first host to create a GlusterFS volume with replica of 3 by providing all hosts and their respective brick directories path. Below command will create a replicated volume across 3 hosts which is most common and preferred FlusterFS volume type. There are other types of volume which we ‘ll discuss in details in another post.

$ sudo gluster volume create glsfsvolume replica 3 glsfshost01:/glsvol01/brick01 glsfshost02:/glsvol01/brick01 glsfshost03:/glsvol01/brick01 

Start the GlusterFS volume once it is created successfully.

$ sudo gluster volume start glsfsvolume

To verify the status and check the information details about the GlusterFS Volume run the below commands.

$ sudo gluster volume status
Status of volume: glsfsvolume
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick glsfshost01:/glsvol01/brick01         56998     0          Y       2945
Brick glsfshost02:/glsvol01/brick01         60035     0          Y       3196
Brick glsfshost03:/glsvol01/brick01         59640     0          Y       3199
Self-heal Daemon on localhost               N/A       N/A        Y       2962
Self-heal Daemon on glsfshost03             N/A       N/A        Y       3216
Self-heal Daemon on glsfshost02             N/A       N/A        Y       3213
Task Status of Volume glsfsvolume
------------------------------------------------------------------------------
There are no active volume tasks

In addition to status command, below command is useful to get information of GlusterFS volume and associated bricks.

$ sudo gluster volume info glsfsvolume
Volume Name: glsfsvolume
Type: Replicate
Volume ID: befe3595-29dc-48ff-aed9-03f432e8a2c9
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: glsfshost01:/glsvol01/brick01
Brick2: glsfshost02:/glsvol01/brick01
Brick3: glsfshost03:/glsvol01/brick01
Options Reconfigured:
cluster.granular-entry-heal: on
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

Congratulations! GlusterFS installation on three Ubuntu hosts step by step completed and we are also able to create a distributed file system that provides scalability, high availability, and fault tolerance.

Conclusion

In summary, we explored GlusterFS installation on Ubuntu operating system with practical demonstration and make sure to provide easy to understand explanation of the same . It is also simple and very resilient storage with a lot of capabilities for various applications and workloads. GlusterFS volume can be mounted and accessed through GlusterFS native,FUSE, NFS, SMB/CIFS and Object storage clients. It’s also important to consider the security aspects and fine-tune the configuration according to your specific requirements before deploying GlusterFS in a production environment. We ‘ll surely, explore client-side configuration and security aspects and performance tuning on GlusterFS in upcoming articles in details.