There is a new version of this tutorial available for Debian 12 (Bookworm).

How to Install Prometheus System Monitoring Tool on Debian 11

Prometheus is a free, open-source and web-based monitoring application that collects metrics from your services and stores them in a time-series database. Prometheus default configuration only exports metrics about itself. But, you can extend it by installing exporters, and other programs. It supports a multi-dimensional data model, multiple modes of graphing and dashboarding.

In this post, we will show you how to install Prometheus monitoring on Debian 11.

Prerequisites

  • A server running Debian 11.
  • A root password is configured on the server.

Install Prometheus

Before starting, it is recommended to create a dedicated user and group for Prometheus. You can create it using the following command:

groupadd --system prometheus
useradd -s /sbin/nologin --system -g prometheus prometheus

Next, download the latest version of Prometheus by running the following command:

curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest|grep browser_download_url|grep linux-amd64|cut -d '"' -f 4|wget -qi -

Once the Prometheus is downloaded, you can see the downloaded file using the following command:

ls -l

You should see the following output:

-rw-r--r-- 1 root root 72638078 Oct  5 16:46 prometheus-2.30.3.linux-amd64.tar.gz

Next, extract the downloaded file using the following command:

tar -xvf prometheus*.tar.gz

Next, change the directory to the extracted directory with the following command:

cd prometheus-2.30.3.linux-amd64

Next, create some required directories using the following command:

mkdir /etc/prometheus
mkdir /var/lib/prometheus

Next, copy the required configuration files and tools with the following commands:

mv prometheus.yml /etc/prometheus/prometheus.yml
mv consoles/ console_libraries/ /etc/prometheus/
mv prometheus promtool /usr/local/bin/

Once you are done, you can proceed to the next step.

Create a Systemd Service File for Prometheus

Next, you will need to create a systemd service file to manage the Prometheus service. You can create it by running the following command:

nano /etc/systemd/system/prometheus.service

Add the following lines:

[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.external-url=

SyslogIdentifier=prometheus
Restart=always

[Install]
WantedBy=multi-user.target

Save and close the file then set proper ownership and permission to the Prometheus configuration directory:

chown -R prometheus:prometheus /etc/prometheus/
chmod -R 775 /etc/prometheus/
chown -R prometheus:prometheus /var/lib/prometheus/

Next, reload the systemd daemon to apply the changes:

systemctl daemon-reload

Next, start the Prometheus service and enable it to start at system reboot using the following command:

systemctl start prometheus
systemctl enable prometheus

You can check the status of the Prometheus with the following command:

systemctl status prometheus

You will get the following output:

? prometheus.service - Prometheus
     Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-10-16 14:06:39 UTC; 4s ago
       Docs: https://prometheus.io/docs/introduction/overview/
   Main PID: 18415 (prometheus)
      Tasks: 5 (limit: 2341)
     Memory: 19.6M
        CPU: 79ms
     CGroup: /system.slice/prometheus.service
             ??18415 /usr/local/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus --web.cons>

Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.172Z caller=head.go:513 component=tsdb msg="On-disk memory mappa>
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.172Z caller=head.go:519 component=tsdb msg="Replaying WAL, this >
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.174Z caller=head.go:590 component=tsdb msg="WAL segment loaded" >
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.174Z caller=head.go:596 component=tsdb msg="WAL replay completed>
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.177Z caller=main.go:849 fs_type=EXT4_SUPER_MAGIC
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.177Z caller=main.go:852 msg="TSDB started"
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.178Z caller=main.go:979 msg="Loading configuration file" filenam>
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.178Z caller=main.go:1016 msg="Completed loading of configuration>
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.179Z caller=main.go:794 msg="Server is ready to receive web requ>
Oct 16 14:06:39 debian11 prometheus[18415]: level=info ts=2021-10-16T14:06:39.179Z caller=tls_config.go:191 component=web msg="TLS is disabled>

By default, Prometheus listens on port 9090. You can check it using the following command:

ss -antpl | grep 9090

You should see the following output:

LISTEN 0      4096               *:9090            *:*    users:(("prometheus",pid=18415,fd=7))

Once you are finished, you can proceed to the next step.

Configure Nginx as a Reverse Proxy for Prometheus

Next, it is recommended to install and configure Nginx as a reverse proxy for Prometheus. First, install the Nginx web server package using the following command:

apt-get install nginx -y

Once Nginx is installed, create an Nginx virtual host configuration file with the following command:

nano /etc/nginx/conf.d/prometheus.conf

Add the following lines:

server {
    listen 80;
    server_name prometheus.example.com;
    location / {
        proxy_pass http://localhost:9090;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

}

Save and close the file then verify the Nginx for any syntax error with the following command:

nginx -t

If everything is fine, you will get the following output:

nginx: configuration file /etc/nginx/nginx.conf test is successful

Finally, restart the Nginx service to apply the changes:

systemctl restart nginx

You can also verfiy the status of the Nginx service using the following command:

systemctl status nginx

You should get the following output:

? nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-10-16 14:08:15 UTC; 2s ago
       Docs: man:nginx(8)
    Process: 18792 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 18793 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 18794 (nginx)
      Tasks: 2 (limit: 2341)
     Memory: 2.5M
        CPU: 35ms
     CGroup: /system.slice/nginx.service
             ??18794 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             ??18795 nginx: worker process

Oct 16 14:08:15 debian11 systemd[1]: Starting A high performance web server and a reverse proxy server...
Oct 16 14:08:15 debian11 systemd[1]: nginx.service: Failed to parse PID from file /run/nginx.pid: Invalid argument
Oct 16 14:08:15 debian11 systemd[1]: Started A high performance web server and a reverse proxy server.

At this point, Nginx is installed and configured to serve the Prometheus. You can now proceed to the next step.

Access Prometheus Dashboard

Now, open your web browser and access the Prometheus dashboard using the URL http://prometheus.example.com. You should see the following page:

Prometheus GUI

Install and Configure node_exporter

node_exporter is an exporter that will monitor and get the metric of the Prometheus server. First, download the latest version of node_exporter using the following command:

wget https://github.com/prometheus/node_exporter/releases/download/v1.2.2/node_exporter-1.2.2.linux-amd64.tar.gz

Once the download is completed, extract the downloaded file with the following command:

tar -xvzf node_exporter-1.2.2.linux-amd64.tar.gz

Next, move the extracted directory to the /etc/prometheus/ directory:

mv node_exporter-1.2.2.linux-amd64 /etc/prometheus/node_exporter

Next, set proper ownership with the following command:

chown -R prometheus:prometheus /etc/prometheus/node_exporter

Next, create a systemd service file to manage the node_exporter service:

nano /etc/systemd/system/node_exporter.service

Add the following lines:

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
ExecStart=/etc/prometheus/node_exporter/node_exporter

[Install]
WantedBy=default.target

Save and close the file then reload the systemd daemon to apply the changes:

systemctl daemon-reload

Next, start the node_exporter service and enable it to start at system reboot:

systemctl start node_exporter
systemctl enable node_exporter

You can check the status of the node_exporter with the following command:

systemctl status node_exporter

You will get the following output:

? node_exporter.service - Node Exporter
     Loaded: loaded (/etc/systemd/system/node_exporter.service; disabled; vendor preset: enabled)
     Active: active (running) since Sun 2021-10-17 05:38:20 UTC; 4s ago
   Main PID: 513 (node_exporter)
      Tasks: 4 (limit: 2341)
     Memory: 4.7M
        CPU: 11ms
     CGroup: /system.slice/node_exporter.service
             ??513 /etc/prometheus/node_exporter/node_exporter

Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=thermal_zone
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=time
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=timex
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=udp_queues
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=uname
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=vmstat
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=xfs
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.382Z caller=node_exporter.go:115 collector=zfs
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.383Z caller=node_exporter.go:199 msg="Listening on" address=:91>
Oct 17 05:38:20 debian11 node_exporter[513]: level=info ts=2021-10-17T05:38:20.383Z caller=tls_config.go:191 msg="TLS is disabled." http2=false

By default, node_exporter listens on port 9100. You can check it with the following command:

ss -antpl | grep 9100

You should get the following output:

LISTEN 0      4096               *:9100            *:*    users:(("node_exporter",pid=513,fd=3))

Add node_exporter to the Prometheus Server

Next, you will need to add the node_exporter to the Prometheus configuration file. You can do it by editing the Prometheus default configuration file:

nano /etc/prometheus/prometheus.yml

Under the 'scrape_config' line, add new job_name node_exporter by adding the following lines.

  - job_name: 'node_exporter'
    static_configs:
    - targets: ['localhost:9100']

Save and close the file then restart the Prometheus service to apply the changes:

systemctl restart prometheus

Verify Prometheus and node_exporter

Now, go to the Prometheus dashboard and click on the Status => Target. You should see your node_exporter in the following screen:

Now, go back to the Prometheus home page and type node_memory_MemAvailable_bytes in the query field and click the 'Execute' button. You will get the following result:

Node memory available

You can also get the node_exporter metrics data using the URL http://your-server-ip:9100/metrics as shown below:

node_exporter metrics

Conclusion

Congratulations! you have successfully installed Prometheus with node_exporter on Debian 11. Feel free to ask me if you have any questions. For more information, visit the official Prometheus documentation.

Share this page:

1 Comment(s)