Prometheus/Grafana

Thank you !!

What ports do you have open on block, relay1, relay 2 ?

i set ins on all 3 for my monitor

on server node edit your prometheus yaml file
here’s mine:

# Here it's Prometheus itself.
scrape_configs:
  - job_name: 'BP_cnode'
    static_configs:
    - targets: ['PRODUCER_IP:12798']
      labels:
        instance: "Producer"
  - job_name: 'BP_node_exporter'
    static_configs:
    - targets: ['PRODUCER_IP:9091']
      labels:
        instance: "Producer"
  - job_name: 'Relay1_cnode'
    static_configs:
    - targets: ['RELAY_1_NODE_EXPORTER_IP:12798']
      labels:
        instance: "Relay1"
  - job_name: 'Relay1_node_exporter'
    static_configs:
    - targets: ['RELAY_1_NODE_EXPORTER_IP:9091']
      labels:
        instance: "Relay1"
  - job_name: 'Relay2_cnode'
    static_configs:
    - targets: ['127.0.0.1:12798']
      labels:
        instance: "Relay2"
  - job_name: 'Relay2_nexporter'
    static_configs:
    - targets: ['127.0.0.1:9091']
      labels:
        instance: "Relay2"

Relay 2 is the server for grafana in my case

  1. on Relay 2 I have opened:

9090/tcp ALLOW 127.0.0.1
12978/tcp ALLOW 127.0.0.1
9091/tcp ALLOW 127.0.0.1

  1. on my other nodes I opened in FW to accept connections from my server to ports 12798 and 9091 (from grafana node server)

you can try and after, restart the services

sudo systemctl restart grafana
sudo systemctl status grafana

sudo systemctl restart prometheus
sudo systemctl status prometheus

and you will use {{instance}} in grafana in order to see your node’s names in grafana…
you cand edit instance string whit the name you will want to see in grafana

2 Likes

are you using only node exporter on relay 1 -3 and block producer?

Yes, only node exporter

That’s why I told u to watch that video… to understand how statistics are send

scrape_configs:

The job name is added as a label job=<job_name> to any timeseries scraped f>

  • job_name: ‘prometheus’

    static_configs:

    • targets: [‘x.x.x.x:9100’] Block Producer Public IP
    • targets: [‘x.x.x.x:12798’] Block Producer Public IP
      labels:
      alias: ‘producer’
      type: ‘cardano-node’
    • targets: [‘localhost:12798’]
      labels:
      alias: ‘monitor’
    • targets: [‘x.x.x.x:9100’] Relay 1 Public IP
    • targets: [‘x.x.x.x:12798’] Relay 1 Public IP
      labels:
      alias: ‘relay1’
      type: ‘cardano-node’
    • targets: [‘localhost:12798’]
      labels:
      alias: ‘monitor’
    • targets: [‘x.x.x.x:9100’] Relay 2 Public IP
    • targets: [‘x.x.x.x:12798’] Relay 2 Public IP
      labels:
      alias: ‘relay2’
      type: ‘cardano-node’
    • targets: [‘localhost:12798’]
      labels:
      alias: ‘monitor’

Block Producer (mainnet-config.json)
],
“hasEKG”: 12788, (Block Producer Public IP)
“hasPrometheus”: [
“x.x.x.x”,
12798
],

Relay 1 (mainnet-config.json)
],
“hasEKG”: 12788, (Relay 1 Public IP)
“hasPrometheus”: [
“x.x.x.x”,
12798
],

Relay 2 (mainnet-config.json)
],
“hasEKG”: 12788, (Relay 2 Public IP)
“hasPrometheus”: [
“x.x.x.x”,
12798
],

sudo ufw status (Block Producer)

9100/tcp ALLOW x.x.x.x (Monitor server private IP)
12798/tcp ALLOW x.x.x.x (Monitor server private IP)

sudo ufw status (Relay 1)

9100/tcp ALLOW x.x.x.x (Monitor server private IP)
12798/tcp ALLOW x.x.x.x (Monitor server private IP)

sudo ufw status (Relay 2)

9100/tcp ALLOW x.x.x.x (Monitor server private IP)
12798/tcp ALLOW x.x.x.x (Monitor server private IP)

sudo ufw status (Monitor)

3000/tcp ALLOW Anywhere
9090/tcp ALLOW Anywhere

I got it working, I had to get rid of my extra server, and just use my relay 1 server. nnow it shows all 3 in there.

1 Like

https://snapshot.raintank.io/dashboard/snapshot/DSoaoU6dC03NIfTxN1tKOB2HYT0XBvDK

Well this was working until I restarted now my prometheus.service isnt starting :expressionless:

Could be something wrong in yaml.configuration… check again the file

I think u don’t need to use TAB, only SPACE TAST… when un edit the configuration

I’m missing Processor Usage from my BP on Grafana, everything else is pulled like Memory and Mempool for relays and BP correctly.

Any ideas?

Thank you

go on Producer and type

sudo systemctl status node_exporter
should be running, if not run again the script (./node_exporter.sh)

cheers,

Its running fine, as mentioned I can see the RAM usage from the BP and all the other fields, just not the CPU. Weird!

then check if u are exporting the metric

curl localhost:12798/metrics | grep cpu_

Looks like it yes:

image

and for memory

image