How to Install Fleet Osquery Manager on Ubuntu 22.04

Fleet is an open-source osquery manager that can be used to maintain secure workstations and servers, and keep an accurate inventory of all your devices. Fleet enables programmable live queries, streaming logs, and real-time visibility of servers, containers, and devices such as laptops and local computers.

With a fleet, you can identify vulnerabilities on your devices. This means that the fleet will automatically identify outdated, vulnerable, or compromised software, apps, and packages. Also, the fleet will identify misconfigurations of devices and MDM enrollment issues. Fleet can be useful for IT industries, security, or any compliance monitoring devices.

Fleet also enables and automates security workflows in a single application. You can collect events using osquery/agent from multiple servers and devices, then store gathered data in a single place that can be accessed via Fleet dashboards or using a terminal via fleetctl.

In this tutorial, you will install Fleet Osquery Manager on an Ubuntu 22.04 server. This process includes the installation of MySQL and Redis servers on an Ubuntu system. And in the end, you'll also learn how to add the host to the fleet via Orbit, which is an osquery runtime and auto-updater that allows you easily deploy osquery, and manage configurations.

Prerequisites

To start with this tutorial, you must have the following requirements:

• A server running Ubuntu 22.04 - This example uses an Ubuntu server with the hostname 'fleet'.
• A non-root user with sudo/root administrator privileges.
• A domain name pointed to the server IP address - This example uses the domain name 'fleet.howtoforge.local'.

That's it; Now let's start the fleet osquery manager installation.

Installing MySQL Server

In the basic architecture, the fleet osquery manager used a MySQL server as the backend database and stored data. The first thing you must do for this guide installs the MySQL server, secure the MySQL installation, and set up the MySQL root password.

To start, run the below apt command to update and refresh your Ubuntu package index.

sudo apt update

Then install the MySQL server package via the following apt command. When prompted, input y to confirm and press ENTER to proceed.

sudo apt install mysql-server

After MySQL is installed, run the below systemctl command to verify the MySQL server and ensure that the service is enabled and running.

sudo systemctl is-enabled mysql
sudo systemctl status mysql

You should receive an output like this - The MySQL server is enabled and will start automatically upon the system startup. And the status of the MySQL server is running.

Next, log in to the MySQL shell via the 'mysql' command below.

sudo mysql

Run the below query to change the password for the MySQL 'root' user. Also, be sure to change the following password in the query. Then, type 'exit' to log out from the MySQL shell.

ALTER USER "root"@"localhost" IDENTIFIED WITH mysql_native_password BY "toor?p4ssw0rd";
exit

With that, you can now start securing the MySQL server via the 'mysql_secure_installation' command below.

sudo mysql_secure_installation

When prompted for MySQL root password, input the new password that you've configured. Then, you'll now be asked about the following MySQL server configurations:

• Set up VALIDATE PASSWORD component on MySQL? Input Y to confirm.
• Input the number password policy that you want to use. Select your preferred choice policy for your MySQL server.
• Change the MySQL root password? Input n for No.
• Remove default MySQL anonymous user? Input Y.
• Disable remote login for MySQL root user? Input Y.
• Remove default database test from MySQL server? Input Y.
• Reload table privileges to apply changes? Input Y to confirm.

With this, the MySQL server is installed, the root password is configured, and the MySQL server is also secured via the 'mysql_secure_installation' command.

Installing Redis Server

By default, the fleet osquery manager uses Redis to ingest and queue the results of distributed queries, cache data, etc. In this step, you'll install Redis on your Ubuntu server.

Run the following apt command to start the Redis installation. When prompted, input y to confirm and press ENTER to proceed.

sudo apt install redis

After Redis is installed, run the following systemctl command to verify the Redis service and ensure that the service is enabled and running.

sudo systemctl is-enabled redis-server
sudo systemctl status redis-server

You should receive an output like this - The output 'enabled' confirms that the Redis service is enabled and will be run automatically upon the system startup. The status of the Redis service by default is running.

Setting up MySQL Database and User

After installing the MySQL server and Redis, you will now create a new database and user that will be used by fleet.

Log in to MySQL shell via the 'mysql' command below.

mysql -u root -p

Run the following queries to create a new MySQL database and user. In this example, you'll create the database fleetdb with the user fleetadmin that will be used for the flee osquery manager installation. Also, be sure to change the password in the following query.

CREATE DATABASE fleetdb;
CREATE USER fleetadmin@localhost IDENTIFIED BY 'S3curep4ssw0rd--=';
GRANT ALL PRIVILEGES ON fleetdb.* TO fleetadmin@localhost WITH GRANT OPTION;
FLUSH PRIVILEGES;

Next, run the following query to verify the list of users on the MySQL server. And you should receive an output that the fleetadmin user added to MySQL.

SELECT USER,host FROM mysql.user;

Run the following query to check privileges for the MySQL user fleetadmin. You should receive an output that the fleetadmin has privileges to access the fleetdb database.

SHOW GRANTS FOR fleetadmin@localhost;

Now input 'quit' to log out from the MySQL shell. And now, you've finished the MySQL server configuration and are ready to install fleet osquery manager.

Downloading Fleet Osquery Manager

Fleet osquery manager is available as a single binary file that provides the following:

• The Fleet TLS web server (no external web server is required but it supports a proxy if desired)
• The Fleet web interface
• The Fleet application management REST API
• The Fleet osquery API endpoints

As for the fleetctl, it's the command-line interface of the fleet that allows you to manage fleet deployment, configurations, integration, and reporting from the command line.

In this step, you'll download the fleet and fleetctl binary package from the official GitHub page. At the time of this writing, the latest version of fleet and fleetctl is v4.26.

Before you begin, create a new system user 'fleet' using the below command.

sudo useradd -r -d /opt/fleet -s /usr/sbin/nologin fleet

Download the fleet binary package and fleetctl - the command-line interface for the fleet - via the curl command below.

curl -LO https://github.com/fleetdm/fleet/releases/download/fleet-v4.26.0/fleet_v4.26.0_linux.tar.gz
curl -LO https://github.com/fleetdm/fleet/releases/download/fleet-v4.26.0/fleetctl_v4.26.0_linux.tar.gz

Once downloaded, extract both fleet and fleetctl packages via the tar command below.

tar xf fleet_v4.26.0_linux.tar.gz
tar xf fleetctl_v4.26.0_linux.tar.gz

Now move the binary file of 'fleet' and 'fleetctl' to the '/usr/local/bin' directory.

cp fleet_v4.26.0_linux/fleet /usr/local/bin/
cp fleetctl_v4.26.0_linux/fleetctl /usr/local/bin/

Check the current PATH environment variable on your system. If the '/usr/local/bin' directory is available on the PATH environment variable, you can then run the 'fleet' and 'fleetctl' commands.

echo $PATH

Verify the full path of the 'fleet' and 'fleetctl' commands using the below command. Both binary files should be available in the '/usr/local/bin' directory.

which fleet
which fleetctl

Verify the version of 'fleet' and 'fleetctl' using the following command. In this example, you've installed fleet and fleetctl v4.26.

fleet version
fleetctl --version

Lastly, run the following 'fleet' command to initialize the database for your installation. Also, be sure to change the details database name, user, and password. With this, you'll create the necessary tables for the fleet osquery manager.

fleet prepare db --mysql_address=127.0.0.1:3306 --mysql_database=fleetdb --mysql_username=fleetadmin --mysql_password=S3curep4ssw0rd--=

Below is the output during the initialization/migration process of the fleet database.

When initialization is finished, you should get an output such as 'Migrations completed'.

Now that the database for fleet osquery manager is migrated. In the next step, you will configure the fleet deployment.

Configuring Fleet Osquery Manager

To deploy fleet, you must ensure that you have verified SSL/TLS certificates. So, before starting, ensure that you have a domain name or local domain name pointed to the server IP address and generated SSL/TLS certificates. You can use letsencrypt or any certificate manager in your local environment.

For this example, generated SSL/TLS certificates from Letsencryopt will be used for the Fleet Osquery Manager installation.

Run the below command to create new directories '/etc/fleet', '/etc/fleet/certs', and a new config file '/etc/fleet/fleet.yml', and the new service file '/etc/systemd/system/fleet.service'.

mkdir -p /etc/fleet/certs
touch /etc/fleet/fleet.yml
touch /etc/systemd/system/fleet.service

Copy your SSL/TLS certificates to the '/etc/fleet/certs' directory.

cp /etc/letsencrypt/live/fleet.howtoforge.local/fullchain.pem /etc/fleet/certs/
cp /etc/letsencrypt/live/fleet.howtoforge.local/privkey.pem /etc/fleet/certs/

Change the ownership of the fleet configuration directory '/etc/fleet/certs' to the user and group 'fleet'.

sudo chown -R fleet:fleet /etc/fleet

Next, open the fleet config file '/etc/fleet/fleet.yml' using the following nano editor command.

nano /etc/fleet/fleet.yml

Add the following lines to the file. Also, be sure to change the database details (dbname, user, and password) in the below lines.

mysql:
  address: 127.0.0.1:3306
  database: fleetdb
  username: fleetadmin
  password: S3curep4ssw0rd--=
redis:
  address: 127.0.0.1:6379
server:
  cert: /etc/fleet/certs/fullchain.pem
  key: /etc/fleet/certs/privkey.pem
logging:
  json: true
# auth:
# jwt_key: 0iXLJRKhB77puDm13G6ehgkClK0kff6N

Save and exit the file '/etc/fleet/fleet.yml' when finished.

Now open the fleet service file '/etc/systemd/system/fleet.service' using the below nano editor command.

sudo nano /etc/systemd/system/fleet.service

Add the following lines to the file. With this, you'll run fleet as a systemd service which allows you easily to manage fleet via the systemctl command utility.

[Unit]
Description=Fleet Osquery Fleet Manager
After=network.target

[Service]
User=fleet
Group=fleet
LimitNOFILE=8192
ExecStart=/usr/local/bin/fleet serve -c /etc/fleet/fleet.yml
ExecStop=/bin/kill -15 $(ps aux | grep "fleet serve" | grep -v grep | awk '{print$2}')

[Install]
WantedBy=multi-user.target

Save and exit the file when finished.

Next, run the below systemctl command to reload the systemd manager and apply the changes.

sudo systemctl daemon-reload

After that, start and enable fleet service using the below systemctl command utility. And the fleet service will be running and enabled.

sudo systemctl start fleet
sudo systemctl enable fleet

Run the below systemctl command utility to verify the fleet service.

sudo systemctl status fleet

The output 'active (running)' confirms that the fleet service is running, and the output '...; enabled;..' confirms that the fleet service will start automatically upon the system startup. Also, you can see the start command of fleet that runs with the config file '/etc/fleet/fleet.yml'.

Configuring UFW Firewall

In this step, you'll set up and run UFW firewall on Ubuntu. You'll open the OpenSSH service port and the TCP port 8080 that is used by fleet osquery manager, then you'll start and enable UFW.

Run the below ufw command to add the OpenSSH service and the TCP port 8080 to the UFW. The output 'Rules update' confirm that the new configuration added to UFW.

sudo ufw allow OpenSSH
sudo ufw allow 8080/tcp

Next, run the following command to start and enable the UFW firewall. When prompted, input y and press ENTER to proceed. With this, the UFW firewall should be running and enabled.

sudo ufw enable

Output:

Verify the status of the UFW firewall using the following command. You should see an output such as 'Status: active' which confirms that UFW is running and enabled. Also, you will see the OpenSSH service port 8080/tcp is available and added to the UFW firewall.

sudo ufw status

Output:

Configuring Fleet Osquery Manager

In this step, you will set up the fleet osquery manager deployment. You will be setting up the first user and setting up the deployment via the web browser.

Open your web browser and visit the domain of your fleet osquery manager installation with TCP port 8080 (i.e: https://fleet.howtoforge.local:8080/).

In the first step, you will be asked to set up the first user for your fleet deployment. Input your full name, email address, and password, then click Next.

Input details organization, then click Next again.

For the fleet URL, you can leave it as default and click Next.

Recheck your fleet configurations and click Confirm to complete the deployment.

When successful, you should get the fleet administration dashboard.

Now the fleet osquery manager installation is finished. With this, you can add new hosts to fleet via multiple ways such as using Orbit (osquery runtime), fleet Desktop for hosts with the desktop environment (including Windows and macOS), or manually by generating fleet secret and TLS certificate.

Adding Hosts via Orbit Osquery Runtime

Orbit is an osquery runtime and auto-updater that allows you easily deploy osquery and manage configurations. Orbit is an agent for fleet, it can be used with or without Fleet, and Fleet can be used with or without Orbit.

In this step, you'll learn how to generate Orbit package installer for Debian-based distributions. Then, you will learn how to add a new host to fleet via Orbit.

To start, move back to the fleet dashboard and click 'Add Hosts'.

Select the 'Advanced' tab, download the fleet certificate 'fleet.pem', then copy the command that will be used to create an orbit package for specific distributions. You can generate an orbit installer for RPM, DEB, and pkg (for macOS).

Next, upload the fleet certificate that you have downloaded to the fleet server. In this example, you will be using the 'scp' to upload the 'fleet.pem' file to the fleet server.

scp fleet.pem [email protected]:/opt/

After the fleet.pem certificate is uploaded, run the command line that will be used to generate the orbit installer package. Be sure to change the parameter '--type' to your preferred package.

In this example, you'll generate an orbit package for Debian-based distribution. For RPM based, you can change the parameter '--type' to 'rpm', while the 'pkg' package can be used for generating an orbit installer for macOS.

fleetctl package --type=deb --fleet-url=https://fleet.howtoforge.local:8080 \
--enroll-secret=TSSnHvy350wlo1HmIeLcxRS3DrQO+Vah \
--fleet-certificate=/opt/fleet.pem

Output:

Once the process is finished, you can see the file 'fleet-osquery_version.deb' in your current working directory.

Next, install the generated orbit package via the dpkg command below. Once installed, the orbit package will create a new service file 'orbit.service' that allows you to manage orbit via systemctl.

sudo dpkg -i fleet-osquery_version.deb

Output:

After orbit installed, run the below systemctl command to start the orbit service. Then, verify the status to ensure that the orbit service is running.

sudo systemctl start orbit
sudo systemctl status orbit

You should receive an output like this - The orbit service is 'active (running)' and it's enabled and will be run automatically upon the system startup.

Now back to the fleet dashboard and you should see the new host 'fleet.howtoforge.local' added to fleet osquery manager.

Click on the hostname 'fleet.howtoforge.local' to get details information about the host.

With this, you've now added a host to fleet osquery manager via Orbit osquery runtime. You've also generated an installer of orbit for Debian-based distributions.

Setting Up Fleetctl for Managing Fleet

Fleetctl or Fleet control is a command line for managing fleet deployment from the terminal. Fleetctl allows you to manage configurations, and queries, generate an osquery installer and enable GitOps workflow with fleet.

In this step, you'll set up fleetctl and connect to the fleet osquery manager that you've installed.

First, run the following command to set up the default fleet URL. Be sure to change the domain name and ensure that you're using an HTTPS secure connection. With this, you'll set up fleet connection in the 'default' context/profile.

fleetctl config set --address https://fleet.howtoforge.local:8080

Log in to your fleet osquery manager using the command below. Be sure to change the email address in the below command.

fleetctl login --email [email protected]

Now input the password that you're using to log in to fleet dashboard. After successful, you should receive an output such as 'Fleet login successful and context configured!'.

After logging in to fleet, run the following fleetctl command to verify your configurations.

Checking the list of available hosts on fleet.

fleetctl get hosts

Output - You should see the host 'fleet.howtoforge.local' is available on fleet with the osquery v5.7.0.

Checking the list of available users on fleet.

fleetctl get ur

Output - You should see the fleet user that you've created.

With this, you've now configured fleetctl and connected to your fleet deployment. You can now set up hosts, and queries, manage updates, running live queries, from your terminal server.

Conclusion

In this tutorial, you've installed Fleet Osquery Manager on an Ubuntu 220.04 server. You've installed Fleet with MySQL as the database backend and Redis for ingesting queue and cache data. In addition, you've secured Fleet with SSL/TLS certificates and running Fleet as a systemd service that allows you to manage Fleet with the systemctl command utility easily.

Lastly, you've added a host to Fleet via Orbit (osquery runtime) and generated a package installer for Debian-based distributions. Also, you've configured fleetctl and logged in to Fleet so you can manage and configure hosts from your terminal server.

With this in mind, you can now add new hosts to Fleet osquery manage via orbit or manually via plain osqueryd service. Also, you can define new queries for monitoring your hosts, set up vulnerability processing that allows you to detect CVEs via Fleet, and many more. Learn more about Fleet from the Fleet's official documentation.