How to Set up Nginx High Availability with Pacemaker and Corosync on CentOS 7

In this tutorial, we will guide you step by step on how to create a High Availability Cluster Nginx Web Server with Pacemaker, Corosync, and Pcsd. We will create the Active-Passive Cluster or Failover-cluster Nginx web server using Pacemaker on a CentOS 7 system.

Pacemaker is an open source cluster manager software that achieves maximum high availability of your services. It's an advanced and scalable HA cluster manager distributed by ClusterLabs.

Corosync Cluster Engine is an open source project derived from the OpenAIS project under new BSD License. It's a group communication system with additional features for implementing High Availability within applications.

There are some applications for the Pacemaker interfaces. Pcsd is one of the Pacemaker command line interface and GUI for managing the Pacemaker. We can create, configure, or add a new node to the cluster with the pcsd command pcs.

Prerequisites

  • 2 or more CentOS 7 Server
    • web01       10.0.15.10
    • web02       10.0.15.11
    • web03       10.0.15.12
  • Floating IP Address 10.0.15.15
  • Root Privileges

What we will do:

  1. Map the Host File
  2. Install Epel Repository and Nginx
  3. Install and Configure Pacemaker, Corosync, and Pcsd
  4. Create and Configure the Cluster
  5. Disable STONITH and Ignore the Quorum Policy
  6. Add the Floating-IP and Resources
  7. Add Rules to the Cluster
  8. Configure Firewalld
  9. Test the setup

Step 1 - Map the Host File

Note: Run step 1 - 3 on all servers web01, web02 and web03.

The first step we must do is to edit the host file on each server in order to map the hostname of all servers. We've 'web01', 'web02' and 'web03' servers, login to the each server with the ssh account.

ssh [email protected][01,02,03]

Edit the '/etc/hosts' file with vim.

vim /etc/hosts

Paste the following configuration there.

10.0.15.10      web01
10.0.15.11      web02
10.0.15.12      web03

Save and exit.

Now test the hosts' mapping configuration.

ping -c 3 web01
ping -c 3 web02
ping -c 3 web03

Make sure 'web01', 'web02' and 'web03' are mapped to correct IP addresses.

Check server interconnection with ping

Step 2 - Install Epel Repository and Nginx

In this step, we will install the epel repository and then install the Nginx web server. EPEL or Extra Packages for Enterprise Linux repository is needed for installing Nginx packages.

Install EPEL Repository using the following yum command.

yum -y install epel-release

Now install Nginx web server from the EPEL repository.

yum -y install nginx

After the installation is complete, change the default index.html page on each server with a new page.

#Run Command on 'web01'
echo '<h1>web01 - hakase-labs</h1>' > /usr/share/nginx/html/index.html

#Run Command on 'web02'
echo '<h1>web02 - hakase-labs</h1>' > /usr/share/nginx/html/index.html

#Run Command on 'web03'
echo '<h1>web03 - hakase-labs</h1>' > /usr/share/nginx/html/index.html

Install epel repo and nginx

EPEL Repository and Nginx web server are now installed on the system.

Step 3 - Install and configure Pacemaker, Corosync, and Pcsd

Pacemaker, Corosync, and Pcsd are available in the default system repository. So they all can be installed from the CentOS repository using the following yum command.

yum -y install corosync pacemaker pcs

After the installation has been completed, enable all services to launch automatically at system boot using the systemctl commands below.

systemctl enable pcsd
systemctl enable corosync
systemctl enable pacemaker

Now start the pcsd Pacemaker command line interface on all servers.

systemctl start pcsd

Next, create a new password for 'hacluster' user and use the same password for all servers. This user has been created automatically during software installation.

Here's how you configure a password for the 'hacluster' user.

passwd hacluster
Enter new password:

High Availability software stack Pacemaker, Corosync, and Pcsd are installed on to the system.

Install and configure Pacemaker, Corosync, and Pcsd

Step 4 - Create and Configure the Cluster

Note: Run steps 4 - 7 on 'web01' only.

In this step, we will create a new cluster with 3 centos servers. Then configure the Floating IP address and add new Nginx resources.

To create the cluster, we need to authorize all servers using the pcs command and the hacluster user.

Authorize all servers with the pcs command and hacluster user and password.

pcs cluster auth web01 web02 web03
Username: hacluster
Password: [email protected]

Create and Configure the Cluster

Now it's time set up the cluster. Define the cluster name and all servers that will be part of the cluster.

pcs cluster setup --name hakase_cluster web01 web02 web03

Now start all cluster services and also enable them.

pcs cluster start --all
pcs cluster enable --all

Start the cluster

Next, check the cluster status.

pcs status cluster

Check cluster status

Step 5 - Disable STONITH and Ignore the Quorum Policy

Since we're not using the fencing device, we will disable the STONITH. STONITH or Shoot The Other Node In The Head is the fencing implementation on Pacemaker. If you're in production, it's better to enable STONITH.

Disable STONITH with the following pcs command.

pcs property set stonith-enabled=false

Next, for the Quorum policy, ignore it.

pcs property set no-quorum-policy=ignore

Check the property list and make sure stonith and the quorum policy are disabled.

pcs property list

Disable STONITH and Ignore the Quorum Policy

The STONITH and Quorum Policy is disabled.

Step 6 - Add the Floating-IP and Resources

Floating IP is the IP address that can be migrated/moved automatically from one server to another server in the same Data Center. And we've already defined the floating IP address for the Pacemaker High-Availability to be '10.0.15.15'. Now we want to add two resources, the Floating IP address resource with the name 'virtual_ip' and a new resource for the Nginx web server named 'webserver'.

Add the new resource floating IP address 'virtual_ip' using the pcs command as shown below.

pcs resource create virtual_ip ocf:heartbeat:IPaddr2 ip=10.0.15.15 cidr_netmask=32 op monitor interval=30s

Next, add a new resource for the Nginx 'webserver'.

pcs resource create webserver ocf:heartbeat:nginx configfile=/etc/nginx/nginx.conf op monitor timeout="5s" interval="5s"

Make sure you got no error result, then check the resources available.

pcs status resources

You will see two resources 'virtual_ip' and a 'webserver'. New resources for the Floating IP and Nginx web server have been added.

Add the Floating-IP and Resources

Step 7 - Add Constraint Rules to the Cluster

In this step, we will setup High Availability Rules, and will setup resource constraint with the pcs command line interface.

Set the collation constraint for webserver and virtual_ip resources with score 'INFINITY'. Also, setup the webserver and virtual_ip resources as same on all server nodes.

pcs constraint colocation add webserver virtual_ip INFINITY

Set the 'virtual_ip' and 'webserver' resources always on same node servers.

pcs constraint order virtual_ip then the webserver

Next, stop the cluster and then start again.

pcs cluster stop --all
pcs cluster start --all

Add Constraint Rules to the Cluster

Now, check again the resources and you will see their status as 'Started' on the same server 'web01'.

pcs status resources

Check pcs status

virtual_ip and webserver resources have been started on the same server/node 'web01'.

Step 8 - Configure Firewalld

The HA-Cluster will run under the firewall configuration firewalld - install it if you do not have the package.

yum -y install firewalld

Start firewalld and enable it to run automatically every time at system boot using following systemctl commands.

systemctl start firewalld
systemctl enable firewalld

Next, add new services to the firewalld with firewall-cmd commands - add high-availability service, HTTP, and HTTPS services for Nginx.

firewall-cmd --permanent --add-service=high-availability
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https

Configure Firewalld

Reload the firewall configuration, and check all services.

firewall-cmd --reload
firewall-cmd --list-all

Make sure ha service with HTTP and https is there on the list.

Reload firewall and check firewall ruleset

Step 9 - Testing

In this step, we're gonna do some test for the cluster. Test the node status ('Online' or 'Offline'), test the corosync members and status, and then test the high-availability of the Nginx webserver by accessing the Floating IP address.

Test node status with the following command.

pcs status nodes

Testing the setup

All nodes are up 'Online'.

Test the corosync members.

corosync-cmapctl | grep members

You will get Corosync members IP address.

Corosync member IP addresses

Check the Corosync members, and you will see the result as shown below.

pcs status corosync

Corosync status

And lastly, check the webserver High Availability. Open your web browser and type the Floating IP address '10.0.15.15'.

You will see the web page from the 'web01' server.

Website web01

Next, stop the cluster on the 'web01' server with the command below.

pcs cluster stop web01

And check again the page, and you will get the page from the 'web02' server as below.

website web02

Additional:

Check the cluster status with the command below.

pcs status

And you will get the result as shown below.

Check pcs status

Setup of Nginx webserver High Availability with Pacemaker, Corosync, and Pcsd on CentOS 7 server has been completed successfully.

Reference

Share this page:

Suggested articles

1 Comment(s)

Add comment

Comments

From: Sungsoo Kim at: 2018-03-13 00:21:50

Thank you for your blog posting.

It was very helpful to me.

I would like to recommend you use systemd:nginx instead of ocf:heartbeat:nginx when creating web server resource because the former is better and we can still use "systemctl status nginx" to see if nginx is running.

 

Thanks!