There is a new version of this tutorial available for Debian 11 (Bullseye).

How to Monitor Log Files with Graylog v3.1 on Debian 10

Graylog is a free and open-source log management tool based on Java, ElasticSearch and MongoDB. Graylog can be used to collect, index and analyze any server log from a centralized location or distributed location. We can easily monitor any unusual activity for debugging applications and logs using Graylog. Graylog provides a powerful query language, alerting abilities, a processing pipeline for data transformation and much more. We also can extend the functionality of Graylog through a REST API and Add-ons.

At this moment there is no official guide of Graylog v3.1 on Debian 10 yet.

Installation of Graylog v3.1 on Debian 10 is in 9 steps:

  • Step 1 : Update Systems with Debian Backport repositories
  • Step 2 : Install some helper
  • Step 3 : Install headless JAVA runtime v11.00
  • Step 4 : Install MongoDB v4.2, a database to stores the configurations and meta information.
  • Step 5 : Install Elasticsearch-OSS 6.x: It stores all the incoming messages and provide a searching facility.
  • Step 6 : Install Graylog v3.1 - It receives and log from various inputs and provide a web interface for analysis and monitoring.
  • Step 7 : Configure Graylog
  • Step 8 : Test Graylog
  • Step 9 : Login Graylog

Prerequisite

  • A minimal Debian 10. We may refer to this tutorial.
  • Minimum 4 GB RAM, 2 core CPU, and 20GB disks
  • Default Password : KataLaluan
  • Default Secret : SecretRahsiaSecreta
  • root access using "su -", Debian recently changes su command behavior. now 'su' command don't replace PATH. use "su -" instead.

Step 1: Update Systems with Debian Backport

Configure system to use the Debian backports repository

cat > /etc/apt/sources.list << EOF
deb http://ftp.debian.org/debian/ buster main contrib non-free
deb http://security.debian.org/debian-security buster/updates main contrib non-free
deb http://ftp.debian.org/debian/ buster-updates main contrib non-free
deb http://ftp.debian.org/debian buster-backports main contrib non-free
EOF
apt -y update
apt -y dist-upgrade

Step 2 - Install headless Java runtime v11.00

Graylog and Elasticsearch is a Java-based application. So, we will need to install Java to your system. By default, the latest version of Java is available in the Debian 10 default repository. We can install it by just running the following command:

apt -y install apt-transport-https default-jdk

Step 3 - Install some helper

We need to install few useful tools as a helper in the process:

  • GnuPG - an implementation of the OpenPGP standard, to help in key management system
  • wget - a tool to retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols
apt -y install gnupg wget

Step 4 - Install MongoDB v4.2

By default, MongoDB is not available in the Debian 10 default repository. So we need to add MongoDB repository to the system:

apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 4B7C549A058F8B6B
echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/4.2 main" | tee /etc/apt/sources.list.d/latest-mongodb.list
apt -y update
apt install -y mongodb-org

Enable and restart MongoDB services:

systemctl enable mongod.service
systemctl start mongod.service

Step 5 : Install Elasticsearch-OSS 6.x

At this moment, Graylog v3.1 is not support Elasticsearch-OSS 7.x yet

We're going to add the Elasticsearch key and repository to the Debian. With the elasticsearch repository provided by the elastic.co, we're able to install the Elasticsearch by running the following command:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
echo "deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main" | tee -a /etc/apt/sources.list.d/latest-elastic-6.x.list
apt -y update
apt -y install elasticsearch-oss

Configure Elasticsearch for the cluster name

sed -i "s/#cluster.name: my-application/cluster.name: graylog-application/g" /etc/elasticsearch/elasticsearch.yml

Enable and restart Elasticsearch services:

systemctl enable elasticsearch.service
systemctl start elasticsearch.service

Step 6 : Install Graylog v3.1

We going to download a simple Graylog package that help to add Graylog key and configure the Graylog repository

cd /tmp/
wget https://packages.graylog2.org/repo/packages/graylog-3.1-repository_latest.deb
dpkg -i graylog-3.1-repository_latest.deb
apt -y update

Install Graylog by running the following command:

apt -y install graylog-server

Step 7 : Configure Graylog

Hash the password, and copy the hash. "KataLaluan" is the current selected password.

echo 'KataLaluan' | tr -d '\n' | sha256sum | cut -d" " -f1

Add the hashed password in the Graylog configuration file

sed -i "s/^root_password_sha2 =\$/root_password_sha2 = a25d2f6605c9e27f182d39b66a8b527eb7f2360e52b2ccc7614f8ac24e472bef/g" /etc/graylog/server/server.conf

Add the secret in the Graylog configuration file, The minimum length of it is 16 characters.

sed -i "s/^password_secret =\$/password_secret = SecretRahsiaSecreta/g" /etc/graylog/server/server.conf

Allow external access to graylog

sed -i "s/^#http_bind_address = 127.0.0.1:9000/http_bind_address = 0.0.0.0:9000/g" /etc/graylog/server/server.conf

Change the timezone accroding to location

sed -i "s/#root_timezone = UTC/root_timezone = Asia\/Kuala_Lumpur/g" /etc/graylog/server/server.conf

Enable and restart Graylog services:

systemctl enable graylog-server.service
systemctl start graylog-server.service

if the Graylog is behind a router, we must set the router WAN's IP address into the Graylog configuration. It also can be a DNS's A records that point to the same IP address

sed -i '/http_publish_uri =/c\http_publish_uri = http://graylog.howtoforge.com:9000/' /etc/graylog/server/server.conf

Step 8 : Test Graylog

Lets test the Graylog using some primitive commands

apt -y install netcat curl

Here is some sample command to log.

echo "Hello Graylog, let's be friends." | nc -w 1 -u 127.0.0.1 9099

Here is some sample command to get Graylog server API status.

curl -X GET http://localhost:9200
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

Here is some sample command to get Graylog server log.

tail -f /var/log/graylog-server/server.log

Step 9 : Login Graylog

Let used the WebGUI. the URL can be:

  • http://<Local_IP_Address>:9000/
  • http://<Public_IP_Address>:9000/
  • http://<A_Record>:9000/

Sample of the URL

  • http://192.168.0.3:9000/
  • http://104.26.2.165:9000/
  • http://graylog.howtoforge:9000/

After enter the URL in a browser, we should see the following sign-in page, the default username is admin, and the selected password is KataLaluan,

After login, we should see the following Graylog page:

Conclusion

Done, we have successfully installed and configured Graylog 3.1 server on Debian 10. Now we can easily see the logs and analysis of the system logs at the central location. Get more information from the Graylog documentation page. Please comment and feedback if have any question.

Happy logging.

Share this page:

5 Comment(s)

Add comment

Please register in our forum first to comment.

Comments

By: Honza

Thank's for excelent how-to. No problem to install graylog on buster, everything went smoothly.

By: L

Hello

Got some bugs on Last update Debian 10

apt -y install gnupg wget

Reading package lists... Done

Building dependency tree

Reading state information... Done

wget is already the newest version (1.20.1-1.1).

Some packages could not be installed. This may mean that you have

requested an impossible situation or if you are using the unstable

distribution that some required packages have not yet been created

or been moved out of Incoming.

The following information may help to resolve the situation:

 

The following packages have unmet dependencies:

 gnupg : Depends: gpgv (< 2.2.12-1+deb10u1.1~) but 2.2.19-1~bpo10+1 is to be installed

E: Unable to correct problems, you have held broken packages.

Please help to fix

Thank you

By: Kenny Khoo

Checked and Tested with Debian 10.3. Its works, nothing change.Please refer https://youtu.be/8erZ6uM0HXY?t=357 for the success gnupg installation.

Sugguest:-- reinstall a minimal Debian- Ensure root login from user using "su -" instaed of "su"- Revise your repos list. unknow 3rd party repos minght screwed it up.Good luck.

By: base653

Thank you! Went perfectly!!

By: Jeff Silverman

Worked like a champ on Ubuntu 20.04 (Focal).  However, I have a problem that I have not seen addressed.  I want to process log files from nginx, which are more or less the same format that Apache and IIS use.  I do not see anything that looks like what I want to do.  I cannot believe that I am the first person to use nginx and graylog.