How to setup Elasticsearch cluster with 3 Nodes on Ubuntu

Elasticsearch is a no-sql database. It is the distributed search and analytics engine, real-time search and analytics for all types of data. Elasticsearch can efficiently store any type of data be it structured or unstructured text, numerical data and index it in a way that supports fast searches. Elasticsearch provides a simple REST API for managing the cluster and indexing/writing and searching the data.

Elasticsearch is built using Java and includes a bundled version of OpenJDK.

To understand Elasticsearch more in detail, refer its official documentation.

In this article, we will focus only on how to set up a cluster of 3 Nodes.

Pre-requisites

  1. Ubuntu 18.04 LTS

What we will do

  1. Download Elasticsearch
  2. Install Java 1.8
  3. Install and Configure Elasticsearch
  4. Test the Cluster

Download Elasticsearch

For linux, the current version 7.4.2, as of now, of Elasticsearch can be downloaded using the following command on your linux systems.

Download On Each Node (Folder name may differ)

mkdir elastic1
cd elastic1/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-linux-x86_64.tar.gz

If you are looking for a specific version, you can always browse the archives on Elasticsearch site. It is always good to go with the latest version as it contains all the fixes for the issues in the previous versions.

Install Java

Elasticsearch need Java to be available on the system.

To install OpenJDK 8, use the following commands on your Ubuntu 18.04 Server

 On Each Node

sudo apt update
sudo apt install openjdk-8-jdk
java --version

Install and Configure Elasticsearch

Now, it’s time to install Elasticsearch from the archive. 

Extract the package we just downloaded in the previous step using the following command.

Install Elasticsearch On Each Node

tar -zxvf elasticsearch-7.4.2-linux-x86_64.tar.gz

Configure Elasticsearch

Before starting the service, Elasticsearch needs to be configured to work in Cluster mode.

Here we have 3 servers

es-node-1: 10.11.10.62 (Initial master)
es-node-2: 10.11.14.248
es-node-3: 10.11.13.158

Open config/elasticsearch.yml file and add the following in it. (Check for your IPs)

vim config/elasticsearch.yml

Add On Master es-node-1 in config/elasticsearch.yml

#give your cluster a name.
cluster.name: my-cluster

#give your nodes a name (change node number from node to node).
node.name: "es-node-1"

#define node 1 as master-eligible:
node.master: true

#define nodes 2 and 3 as data nodes:
node.data: true

#enter the private IP and port of your node:
network.host: 10.11.10.62
http.port: 9200

#detail the private IPs of your nodes:
discovery.zen.ping.unicast.hosts: ["10.11.10.62", "10.11.14.248","10.11.13.158"]

cluster.initial_master_nodes:
- 10.11.10.62

 

Add On es-node-2 in config/elasticsearch.yml (This node does not contain: cluster.initial_master_nodes)

#give your cluster a name.
cluster.name: my-cluster

#give your nodes a name (change node number from node to node).
node.name: "es-node-2"

#define node 1 as master-eligible:
node.master: true

#define nodes 2 and 3 as data nodes:
node.data: true

#enter the private IP and port of your node:
network.host: 10.11.14.248
http.port: 9200

#detail the private IPs of your nodes:
discovery.zen.ping.unicast.hosts: ["10.11.10.62", "10.11.14.248","10.11.13.158"]

Add On es-node-3 in config/elasticsearch.yml (This node does not contain: cluster.initial_master_nodes)

#give your cluster a name.
cluster.name: my-cluster

#give your nodes a name (change node number from node to node).
node.name: "es-node-3"

#define node 1 as master-eligible:
node.master: true

#define nodes 2 and 3 as data nodes:
node.data: true

#enter the private IP and port of your node:
network.host: 10.11.13.158
http.port: 9200

#detail the private IPs of your nodes:
discovery.zen.ping.unicast.hosts: ["10.11.10.62", "10.11.14.248","10.11.13.158"]

Start and Test the Cluster

Start the Cluster

On Each Node (First start Master es-node1)

Use the following command to start Elasticsearch in foreground

bin/elasticsearch

Note:

Elasticsearch uses a mmapfs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in the following out of memory exceptions.

[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

On Ubuntu 18.04 we can increase the limits by running the following command as root/sudo:

sudo sysctl -w vm.max_map_count=262144

 Add -d to start to start Elasticsearch in background

bin/elasticsearch -d

Once the cluster starts, you can see logs as follows on es-node-1. Notice how nodes are added to es-node-1 when they are started.

[2019-11-30T08:43:05,766][INFO ][o.e.c.s.ClusterApplierService] [es-node-1] added {{es-node-2}{V9-rBKeJRe2S8UPW96_XzA}{65Bu-WtmTj-Hce_lFAK-ng}{10.11.14.248}{10.11.14.248:9300}{dilm}{ml.machine_memory=4135120896, ml.max_open_jobs=20, xpack.installed=true},}, term: 1, version: 18, reason: Publication{term=1, version=18}
[2019-11-30T08:43:30,317][INFO ][o.e.c.s.MasterService    ] [es-node-1] node-join[{es-node-3}{TArASFSyS2-gVcNaH-XosQ}{g9X4hxLBQAu3QBcPXL9JAQ}{10.11.13.158}{10.11.13.158:9300}{dilm}{ml.machine_memory=4135124992, ml.max_open_jobs=20, xpack.installed=true} join existing leader], term: 1, version: 19, reason: added {{es-node-3}{TArASFSyS2-gVcNaH-XosQ}{g9X4hxLBQAu3QBcPXL9JAQ}{10.11.13.158}{10.11.13.158:9300}{dilm}{ml.machine_memory=4135124992, ml.max_open_jobs=20, xpack.installed=true},}
[2019-11-30T08:43:30,820][INFO ][o.e.c.s.ClusterApplierService] [es-node-1] added {{es-node-3}{TArASFSyS2-gVcNaH-XosQ}{g9X4hxLBQAu3QBcPXL9JAQ}{10.11.13.158}{10.11.13.158:9300}{dilm}{ml.machine_memory=4135124992, ml.max_open_jobs=20, xpack.installed=true},}, term: 1, version: 19, reason: Publication{term=1, version=19

Test the cluster

Open a new terminal from which the cluster is reachable and try the following commands

curl http://10.11.10.62:9200/_cluster/stats?pretty

curl http://10.11.10.62:9200/_nodes/process?pretty

curl http://10.11.10.62:9200/_cluster/stats?pretty

Conclusion

In this article, we saw the steps to download the latest version of elasticsearch, install Java 8, configure Elasticsearch, start and test the cluster.

Share this page:

Suggested articles

0 Comment(s)

Add comment