How to take a Backup of an Elasticsearch Cluster

The mechanism of taking backup in Elasticsearch is called Snapshot. A snapshot is a backup taken from an Elasticsearch cluster that is in a running state. There is no need to take down the cluster which helps avoid maintenance windows of the applications. A snapshot of an individual index or of the entire cluster can be taken and stored in a repository on a shared file system.

Snapshots in Elasticsearch are taken incrementally. This means that when it creates a snapshot of an index, Elasticsearch avoids copying any data that is already stored as part of an earlier snapshot of the same index. Therefore it can be efficient to take snapshots of the cluster on a regular basis.

In the same way we can take a backup of the cluster in running state, we can also restore a snapshot into a running cluster. When we restore an index, we can even alter the name of the restored index as well as some of its settings.

To make backups, we must register a snapshot repository before we can perform the snapshot and restore operations. In order to register the shared file system repository for the cluster, it is necessary to mount the same shared file system to the same location on all master and data nodes. This location must be registered in the config file on all master and data nodes.

In this article, we will verify the NFS shared repository, and see the steps to take a snapshot and restore it.

Pre-requisites

  1. NFS Shared directory available and mounted on all 3 Nodes of Elasticsearch on the same location
  2. Elasticsearch Cluster of 3 Nodes on 3 Ubuntu servers.

What we will do

  1. Verify NFS Server Setup.
  2. Verify Elasticsearch Cluster Configuration
  3. Register a Repository to take backups.
  4. Take a backup and restore.

Verify NFS Server/Client Setup.

In this article, we won’t talk about the NFS setup since it does not come under the scope of this article. But to take backup of Elasticsearch we would need the following setup in place.

es-node-1(10.11.10.61)  : NFS Client
es-node-2(10.11.10.62) : NFS Client
es-node-3(10.11.10.63) : NFS Client
NFS Server(10.11.10.64) : NFS Server

Here, 

NFS server has shared its “/home/ubuntu/shared/” directory with Elasticsearch Nodes.

Every Elasticsearch has its local directory “/home/ubuntu/mounted” mounted on  NFS’s shared directory “/home/ubuntu/shared/”. We need to make sure that ownership of all the directories belongs to the same user we would start Elasticsearch with.

Once we have this setup in place, we can proceed further.

Verify Elasticsearch Cluster Configuration

Do the following configurations to setup Elasticsearch to work in Cluster mode:

Here, if you have set up an Elasticsearch cluster you must be aware of the following configuration.

The only configuration we need to make to take Elasticsearch Cluster Backup apart from the existing Elasticsearch Cluster Configuration is "path.repo: ["/home/ubuntu/mounted"]": 

vim config/elasticsearch.yml
path.repo: ["/home/ubuntu/mounted"] 

Keep this same on each node.

Configuration on Node1

#give your cluster a name.
cluster.name: my-cluster
#give your nodes a name (change node number from node to node).
node.name: "es-node-1"
#define node 1 as master-eligible:
node.master: true
#define nodes 2 and 3 as data nodes:
node.data: true
#enter the private IP and port of your node:
network.host: 10.11.10.61
http.port: 9200
#detail the private IPs of your nodes:
discovery.zen.ping.unicast.hosts: ["10.11.10.61", "10.11.10.62", "10.11.10.63"]
cluster.initial_master_nodes:
- 10.11.10.61
path.repo: ["/home/ubuntu/mounted"]

Configuration on Node2

#give your cluster a name.
cluster.name: my-cluster
#give your nodes a name (change node number from node to node).
node.name: "es-node-2"
#define node 2 as master-eligible:
node.master: false
#define nodes 2 and 3 as data nodes:
node.data: true 
#enter the private IP and port of your node:
network.host: 10.11.10.62
http.port: 9200 
#detail the private IPs of your nodes:
discovery.zen.ping.unicast.hosts: ["10.11.10.61", "10.11.10.62", "10.11.10.63"
path.repo: ["/home/ubuntu/mounted"]

Configuration on Node3

#give your cluster a name.
cluster.name: my-cluster
#give your nodes a name (change node number from node to node).
node.name: "es-node-3"
#define node 3 as master-eligible:
node.master: false
#define nodes 2 and 3 as data nodes:
node.data: true
#enter the private IP and port of your node:
network.host: 10.11.10.63
http.port: 9200
#detail the private IPs of your nodes:
discovery.zen.ping.unicast.hosts: ["10.11.10.61","10.11.10.62","10.11.10.63""]
path.repo: ["/home/ubuntu/mounted"]

Backup Elasticsearch cluster

Once you have all this configuration in place, start all the Elasticsearch Nodes, first starting the initial master.

Register a Repository to take backups

Check the existing repositories using the following command.

curl -XGET 'http://IP_Of_Elasticsearch_Node_Or_Master:9200/_snapshot/_all?pretty=true'

If we get blank response, it  indicates that we don’t have any repositories setup yet

 To setup a repository execute the following command.

curl -XPUT 'http://IP_Of_Elasticsearch_Node_Or_Master:9200/_snapshot/my_backup' -d {
"type": "fs",
"settings": {
"location": "/home/ubuntu/mounted",
"compress": true
}
}'

Here, “my_backup” in the above command is the name of the repository.

We can check the repositories registered using the following command

curl -XGET 'http://IP_Of_Elasticsearch_Node_Or_Master:9200/_snapshot/_all?pretty=true'

Backup and Restore of an Elasticsearch Cluster

Take a backup

Once we have created a repo, we are ready to take a backup.

Use the following command to take a backup named “snapshot_name”

curl -XPUT "https://IP_Of_Elasticsearch_Node_Or_Master:9200/_snapshot/my_backup/snapshot_name?wait_for_completion=true"

Restore a backup

The Snapshot we have taken can be restored using the following command.

Use the following command to restore the backup named “snapshot_name”

curl -XPOST "http://IP_Of_Elasticsearch_Node_Or_Master:9200/_snapshot/my_backup/snapshot_name/_restore?wait_for_completion=true"

Conclusion

In this article, we saw the steps to register a repository and take a backup and restore it.

Share this page:

0 Comment(s)