How to Monitor Log Files with Graylog v3.1 on Debian 10
Graylog is a free and open-source log management tool based on Java, ElasticSearch and MongoDB. Graylog can be used to collect, index and analyze any server log from a centralized location or distributed location. We can easily monitor any unusual activity for debugging applications and logs using Graylog. Graylog provides a powerful query language, alerting abilities, a processing pipeline for data transformation and much more. We also can extend the functionality of Graylog through a REST API and Add-ons.
At this moment there is no official guide of Graylog v3.1 on Debian 10 yet.
Installation of Graylog v3.1 on Debian 10 is in 9 steps:
- Step 1 : Update Systems with Debian Backport repositories
- Step 2 : Install some helper
- Step 3 : Install headless JAVA runtime v11.00
- Step 4 : Install MongoDB v4.2, a database to stores the configurations and meta information.
- Step 5 : Install Elasticsearch-OSS 6.x: It stores all the incoming messages and provide a searching facility.
- Step 6 : Install Graylog v3.1 - It receives and log from various inputs and provide a web interface for analysis and monitoring.
- Step 7 : Configure Graylog
- Step 8 : Test Graylog
- Step 9 : Login Graylog
- A minimal Debian 10. We may refer to this tutorial.
- Minimum 4 GB RAM, 2 core CPU, and 20GB disks
- Default Password : KataLaluan
- Default Secret : SecretRahsiaSecreta
- root access using "su -", Debian recently changes su command behavior. now 'su' command don't replace PATH. use "su -" instead.
Step 1: Update Systems with Debian Backport
Configure system to use the Debian backports repository
cat > /etc/apt/sources.list << EOF
deb http://ftp.debian.org/debian/ buster main contrib non-free
deb http://security.debian.org/debian-security buster/updates main contrib non-free
deb http://ftp.debian.org/debian/ buster-updates main contrib non-free
deb http://ftp.debian.org/debian buster-backports main contrib non-free
apt -y update
apt -y dist-upgrade
Step 2 - Install headless Java runtime v11.00
Graylog and Elasticsearch is a Java-based application. So, we will need to install Java to your system. By default, the latest version of Java is available in the Debian 10 default repository. We can install it by just running the following command:
apt -y install apt-transport-https default-jdk
Step 3 - Install some helper
We need to install few useful tools as a helper in the process:
- GnuPG - an implementation of the OpenPGP standard, to help in key management system
- wget - a tool to retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols
apt -y install gnupg wget
Step 4 - Install MongoDB v4.2
By default, MongoDB is not available in the Debian 10 default repository. So we need to add MongoDB repository to the system:
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 4B7C549A058F8B6B
echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/4.2 main" | tee /etc/apt/sources.list.d/latest-mongodb.list
apt -y update
apt install -y mongodb-org
Enable and restart MongoDB services:
systemctl enable mongod.service
systemctl start mongod.service
Step 5 : Install Elasticsearch-OSS 6.x
At this moment, Graylog v3.1 is not support Elasticsearch-OSS 7.x yet
We're going to add the Elasticsearch key and repository to the Debian. With the elasticsearch repository provided by the elastic.co, we're able to install the Elasticsearch by running the following command:
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
echo "deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main" | tee -a /etc/apt/sources.list.d/latest-elastic-6.x.list
apt -y update
apt -y install elasticsearch-oss
Configure Elasticsearch for the cluster name
sed -i "s/#cluster.name: my-application/cluster.name: graylog-application/g" /etc/elasticsearch/elasticsearch.yml
Enable and restart Elasticsearch services:
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
Step 6 : Install Graylog v3.1
We going to download a simple Graylog package that help to add Graylog key and configure the Graylog repository
dpkg -i graylog-3.1-repository_latest.deb
apt -y update
Install Graylog by running the following command:
apt -y install graylog-server
Step 7 : Configure Graylog
Hash the password, and copy the hash. "KataLaluan" is the current selected password.
echo 'KataLaluan' | tr -d '\n' | sha256sum | cut -d" " -f1
Add the hashed password in the Graylog configuration file
sed -i "s/^root_password_sha2 =\$/root_password_sha2 = a25d2f6605c9e27f182d39b66a8b527eb7f2360e52b2ccc7614f8ac24e472bef/g" /etc/graylog/server/server.conf
Add the secret in the Graylog configuration file, The minimum length of it is 16 characters.
sed -i "s/^password_secret =\$/password_secret = SecretRahsiaSecreta/g" /etc/graylog/server/server.conf
Allow external access to graylog
sed -i "s/^#http_bind_address = 127.0.0.1:9000/http_bind_address = 0.0.0.0:9000/g" /etc/graylog/server/server.conf
Change the timezone accroding to location
sed -i "s/#root_timezone = UTC/root_timezone = Asia\/Kuala_Lumpur/g" /etc/graylog/server/server.conf
Enable and restart Graylog services:
systemctl enable graylog-server.service
systemctl start graylog-server.service
if the Graylog is behind a router, we must set the router WAN's IP address into the Graylog configuration. It also can be a DNS's A records that point to the same IP address
sed -i '/http_publish_uri =/c\http_publish_uri = http://graylog.howtoforge.com:9000/' /etc/graylog/server/server.conf
Step 8 : Test Graylog
Lets test the Graylog using some primitive commands
apt -y install netcat curl
Here is some sample command to log.
echo "Hello Graylog, let's be friends." | nc -w 1 -u 127.0.0.1 9099
Here is some sample command to get Graylog server API status.
curl -X GET http://localhost:9200
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
Here is some sample command to get Graylog server log.
tail -f /var/log/graylog-server/server.log
Step 9 : Login Graylog
Let used the WebGUI. the URL can be:
Sample of the URL
After enter the URL in a browser, we should see the following sign-in page, the default username is admin, and the selected password is KataLaluan,
After login, we should see the following Graylog page:
Done, we have successfully installed and configured Graylog 3.1 server on Debian 10. Now we can easily see the logs and analysis of the system logs at the central location. Get more information from the Graylog documentation page. Please comment and feedback if have any question.