Apache Zookeeper is a free and open-source service for distributed systems that helps you to manage a large set of hosts. It is a centralized service used for maintaining configuration information, naming and group services. It is written in Java and allows developers to focus on building software features without worrying about the distributed nature of the application. Apache ZooKeeper most commonly used in "read-dominant" workloads where reads are much more common than writes. It comes with a rich set of features including, Synchronization, Ordered Messages, Serialization, Reliability, Atomicity, Fast Processing and many more. It has a locking and synchronization mechanism that helps you in automatic fail recovery while connecting other distributed applications. Zookeeper project is one of the most successful projects from the Apache foundation. Many companies adopted Zookeeper for creating highly available distributed systems at large scale.
In this tutorial, we will show you how to set up a single-node Apache Zookeeper cluster on Debian 10 server.
Prerequisites
- A server running Debian 10 with 2 GB of RAM.
- A root password is configured on your server.
Getting Started
Before starting, it is recommended to update your server with the latest version. You can update it using the following command:
apt-get update -y
apt-get upgrade -y
Once your server is updated, restart it to implement the changes.
Install Java
Apache Zookeeper is written in the Java language. So you will need to install Java in your system. By default, the latest version of Java is available in the Debian 10 default repository. You can install it by running the following command:
apt-get install default-jdk -y
After installing Java, verify the installed version of Java using the following command:
java --version
You should get the following output:
openjdk 11.0.5 2019-10-15 OpenJDK Runtime Environment (build 11.0.5+10-post-Debian-1deb10u1) OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Debian-1deb10u1, mixed mode, sharing)
Once you are done, you can proceed to create a system user for Zookeeper.
Create a System User for Zookeeper
It is a good idea to create a dedicated user to run the Zookeeper service. This will improve your security and manageability.
First, run the following command to create a zookeeper user with home directory:
useradd zookeeper -m
Next, set the default shell for the zookeeper user using the following command:
usermod --shell /bin/bash zookeeper
Next, set a password for zookeeper user using the following command:
passwd zookeeper
Next, add the zookeeper user to the sudo group:
usermod -aG sudo zookeeper
At this point, you have created and configured the Zookeeper user. You can now proceed to download the Zookeeper.
Download Zookeeper
First, you will need to download the latest version of the Apache Zookeeper from its official website. At the time of writing this article, the latest version of Apache Zookeeper is 3.5.6.
To download it, change the directory to the /opt and run the following command:
cd /opt
wget https://archive.apache.org/dist/zookeeper/stable/apache-zookeeper-3.5.6-bin.tar.gz
Once the download is completed, extract the downloaded file with the following command:
tar -xvzf apache-zookeeper-3.5.6-bin.tar.gz
Next, rename the extracted directory to zookeeper and give proper permissions:
mv apache-zookeeper-3.5.6-bin zookeeper
chown -R zookeeper:zookeeper /opt/zookeeper
Next, you will need to create a data directory for Zookeeper to stores all configuration and state data. You can create in on the local filesystem or the remote filesystem as per your need.
You can create a data directory using the following command:
mkdir -p /data/zookeeper
Next, change the ownership of the data directory to zookeeper user:
chown -R zookeeper:zookeeper /data/zookeeper
At this point, you have downloaded Zookeeper and create a data directory for it. You can now proceed to configure the Zookeeper.
Configure Zookeeper
Zookeeper all the configuration files are located inside /opt/zookeeper/conf/ directory. You can create a Zookeeper configuration file inside /opt/zookeeper/conf/ directory using the following command:
nano /opt/zookeeper/conf/zoo.cfg
Add the following lines:
tickTime=2500 dataDir=/data/zookeeper clientPort=2181 maxClientCnxns=80
Save and close the file when you are finished.
Where:
- tickTime : This option sets the length of a tick in milliseconds.
- dataDir : Specify the data directory to store Zookeeper data.
- clientPort : Specify the port used to listen for client connections.
- maxClientCnxns : Used to limits the maximum number of client connections.
Note : Above parameters are for development and testing. You can change them according to your need.
At this point, you have configured Zookeeper. You can now ready to start the Zookeeper server.
Start and Test Zookeeper Server
You can start the Zookeeper server using the following command:
/opt/zookeeper/bin/zkServer.sh start
You should get the following output:
/usr/bin/java ZooKeeper JMX enabled by default Using config: /opt/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
Next, you can connect to the Zookeeper server on port 2181 using the following command:
/opt/zookeeper/bin/zkCli.sh -server 127.0.0.1:2181
Once connected, you should see the following output:
[zk: 127.0.0.1:2181(CONNECTED) 0]
Next, run the help command to get a list of commands that you can execute from the client.
[zk: 127.0.0.1:2181(CONNECTED) 0] help
You should see the following output:
ZooKeeper -server host:port cmd args addauth scheme auth close config [-c] [-w] [-s] connect host:port create [-s] [-e] [-c] [-t ttl] path [data] [acl] delete [-v version] path deleteall path delquota [-n|-b] path get [-s] [-w] path getAcl [-s] path history listquota path ls [-s] [-w] [-R] path ls2 path [watch] printwatches on|off quit reconfig [-s] [-v version] [[-file path] | [-members serverID=host:port1:port2;port3[,...]*]] | [-add serverId=host:port1:port2;port3[,...]]* [-remove serverId[,...]*] redo cmdno removewatches path [-c|-d|-a] [-l] rmr path set [-s] [-v version] path data setAcl [-s] [-v version] [-R] path acl setquota -n|-b val path stat [-w] path sync path
After testing, exit from the client session using the quit command:
[zk: 127.0.0.1:2181(CONNECTED) 0] quit
You can now stop the Zookeeper service with the following command:
/opt/zookeeper/bin/zkServer.sh stop
You should see the following command:
/usr/bin/java ZooKeeper JMX enabled by default Using config: /opt/zookeeper/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED
At this point, you have successfully started and tested the Zookeeper server.
Create a Systemd Service File for Zookeeper
Next, you will need to create a systemd service file to manage the Zookeeper service using systemd.
You can create it with the following command:
nano /etc/systemd/system/zookeeper.service
Add the following lines:
[Unit] Description=Zookeeper Daemon Documentation=http://zookeeper.apache.org Requires=network.target After=network.target [Service] Type=forking WorkingDirectory=/opt/zookeeper User=zookeeper Group=zookeeper ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/zoo.cfg ExecReload=/opt/zookeeper/bin/zkServer.sh restart /opt/zookeeper/conf/zoo.cfg TimeoutSec=30 Restart=on-failure [Install] WantedBy=default.target
Save and close the file when you are finished. Then, reload the systemd daemon with the following command:
systemctl daemon-reload
Next, start the Zookeeper service and enable it to start after system reboot using the following command:
systemctl start zookeeper
systemctl enable zookeeper
You can now verify the Zookeeper status using the following command:
systemctl status zookeeper
You should see the following output:
? zookeeper.service - Zookeeper Daemon
Loaded: loaded (/etc/systemd/system/zookeeper.service; disabled; vendor preset: enabled)
Active: active (running) since Sun 2019-12-29 15:12:39 UTC; 8min ago
Docs: http://zookeeper.apache.org
Process: 9229 ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS)
Main PID: 9245 (java)
Tasks: 29 (limit: 2359)
Memory: 49.7M
CGroup: /system.slice/zookeeper.service
??9245 java -Dzookeeper.log.dir=/opt/zookeeper/bin/../logs -Dzookeeper.log.file=zookeeper-zookeeper-server-debian10.log -Dzookeeper.
Dec 29 15:12:38 debian10 systemd[1]: Starting Zookeeper Daemon...
Dec 29 15:12:38 debian10 zkServer.sh[9229]: /usr/bin/java
Dec 29 15:12:38 debian10 zkServer.sh[9229]: ZooKeeper JMX enabled by default
Dec 29 15:12:38 debian10 zkServer.sh[9229]: Using config: /opt/zookeeper/conf/zoo.cfg
Dec 29 15:12:39 debian10 zkServer.sh[9229]: Starting zookeeper ... STARTED
Dec 29 15:12:39 debian10 systemd[1]: Started Zookeeper Daemon.
Now, you can easily manage the Zookeeper service with systemd.
Conclusion
Congratulations! you have successfully installed and configured a single node Zookeeper cluster on Debian 10 server. This setup is very useful for developmental and testing environments. I hope you have now enough knowledge to set up a single node Zookeeper cluster. Feel free to ask me if you have any questions.