Prometheus + Grafana on Home LAN
Here is a big picture: Multiple LAN nodes will have node_exporter installed. These are running:
- pfSense (FreeBSD)
- Linux - RHEL/CENTOS 7
- Linux - Ubuntu
- Windows 10
A server (CENTOS) will run Prometheus scraping the nodes and exposing the data to Grafana. The same server (CENTOS) will run Grafana dashboard with cute GUI.
Install node_exporter
node_exporter on pfSense
Available as a module now. To verify the install point your browser to: http://pfsense:9100/metrics
node_exporter CENTOS 7
https://www.howtoforge.com/tutorial/how-to-install-prometheus-and-node-exporter-on-centos-7/
wget https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz
tar -xzvf node_exporter-0.18.1.linux-amd64.tar.gz
mv node_exporter-0.18.1.linux-amd64 /home/prometheus/node_exporter
Then sudo emacs /etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
ExecStart=/home/prometheus/node_exporter/node_exporter
[Install]
WantedBy=default.target
Restart:
sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo netstat -plntu
wmi_exporter on Windows 10
Install wmi_exporter.
https://github.com/martinlindhe/wmi_exporter/releases/tag/v0.7.0
Get https://github.com/martinlindhe/wmi_exporter/releases/download/v0.7.0/wmi_exporter-0.7.0-amd64.msi
Install Prometheus
Prometheus on CENTOS 8
Follow https://computingforgeeks.com/how-to-install-prometheus-on-rhel-8/
sudo groupadd --system prometheus
sudo useradd -s /sbin/nologin --system -g prometheus prometheus
sudo mkdir /var/lib/prometheus
for i in rules rules.d files_sd; do
sudo mkdir -p /etc/prometheus/${i};
done
cd /tmp
export RELEASE=2.10.0
wget https://github.com/prometheus/prometheus/releases/download/v${RELEASE}/prometheus-${RELEASE}.linux-amd64.tar.gz
tar xvf prometheus-${RELEASE}.linux-amd64.tar.gz
cd prometheus-${RELEASE}.linux-amd64/
sudo cp prometheus promtool /usr/local/bin/
sudo cp -r consoles/ console_libraries/ /etc/prometheus/
Edit sudo emacs /etc/prometheus/prometheus.yml
# Global config
global:
scrape_interval: 15s # Default is every 1 minute.
evaluation_interval: 15s # The default is every 1 minute.
scrape_timeout: 15s # scrape_timeout is set to the global default (10s).
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus'
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ['localhost:9090']
sudo vim /etc/systemd/system/prometheus.service
[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
Environment="GOMAXPROCS=2"
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus \
--web.console.templates=/etc/prometheus/consoles \
--web.console.libraries=/etc/prometheus/console_libraries \
--web.listen-address=0.0.0.0:9090 \
--web.external-url=
SyslogIdentifier=prometheus
Restart=always
[Install]
WantedBy=multi-user.target
Set directory permissions:
for i in rules rules.d files_sd; do
sudo chown -R prometheus:prometheus /etc/prometheus/${i};
done
for i in rules rules.d files_sd; do
sudo chmod -R 775 /etc/prometheus/${i};
done
sudo chown -R prometheus:prometheus /var/lib/prometheus/
Start Prometheus service:
sudo systemctl daemon-reload
sudo systemctl start prometheus
systemctl status prometheus.service
Configure firewalld:
sudo firewall-cmd --permanent --add-rich-rule \
'rule family="ipv4" source address="192.168.1.0/24" port protocol="tcp" port="9090" accept'
sudo firewall-cmd --reload
Verify the install by pointing the browser to http://localhost:9090/
To add targets: sudo emacs /etc/prometheus/prometheus.yml
targets: ['localhost:9090', '192.168.1.1:9100']
Then restart prometheus:
sudo systemctl restart prometheus
Install Grafana
Let’s try Grafana Enterprise:
https://grafana.com/docs/grafana/latest/installation/rpm/
Edit /etc/yum.repos.d/grafana.repo:
[grafana]
name=grafana
baseurl=https://packages.grafana.com/enterprise/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
Then yum install grafana-enterprise
.
To start the service and verify that the service has started:
systemctl daemon-reload
systemctl start grafana-server
systemctl status grafana-server
Configure the Grafana server to start at boot: systemctl enable grafana-server