Grafana Dashboard issue

Staking & Delegation Setup a Stake Pool
1

image
image1215×847 185 KB

I can’t understand why my core and relay1 system health doesn’t show up like relay2-host.

Everything else seems to be okay.

Any suggestions that you guys might run into the issue before?

2

First check if u export the metric


curl localhost:12798/metrics

and compare the output for all of 3

3

Everything matched up

4

I see relay 1 and core do have some data. Have you installed node exporter on the relay on core node?

5

Yes, I have installed in all three nodes.

6

Can you try @Alexd1985 idea but with the node exporter port? Also check that your firewall is allowing traffic on those ports

7

Check also the config file… the prometheud IP match for all?


 "hasPrometheus": [
   "127.0.0.1",
   12798
 ],
8

All three nodes has changed to “0.0.0.0”

9

then, if the metrics are ok, the node exporter is up and running and also the ports are opened in FW … perhaps it’s something wrong with the dashboard… check how are the metrics defined in grafana for each node

10

I’d copied the panel setting from relay2-host.

image
image1505×830 69 KB

image
image1500×834 52 KB

11

for relay 2 you have alias relay2-host and for relay 1 you have the alias relay1

check for the alias defined inside the promotheus.yml

12

This is what I have in prometheus.yml

global configs

global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

scrape_timeout is set to the global default (10s).

scrape_configs:

  • job_name: ‘cardano’ # To scrape data from the running cardano-node
    scrape_interval: 15s
    static_configs:

    • targets: [‘relay IP:12798’]
      labels:
      alias: ‘relay1’
      type: ‘cardano-node’
    • targets: [‘core IP:12798’]
      labels:
      alias: ‘core’
      type: ‘cardano-node’
    • targets: [‘127.0.0.1:12798’]
      labels:
      alias: ‘relay2-host’
      type: ‘cardano-node’

  • job_name: ‘node’ # To scrape data from a node exporter to monitor the linux host metrics.
    scrape_interval: 15s
    static_configs:

    • targets: [‘relay IP:9091’]
      labels:
      alias: ‘relay1’
      type: ‘host-system’
    • targets: [‘core IP:9091’]
      labels:
      alias: ‘core’
      type: ‘host-system’
    • targets: [‘127.0.0.1:9091’]
      labels:
      alias: ‘relay2-host’
      type: ‘host-system’
13

it looks fine, wired…

with this command curl localhost:12798/metrics do you see the CPU, RAM, SWAP metric for all 3 nodes right?

try to restart the prometheus service

14

Yes, I see all three nodes

15

I take it ‘Core IP’ and ‘Relay IP’ have valid ip addresses?

Are you running this on a single server or does each node have its own server? Where is Prometheus running?

16

Yes, each node has it own server.
relay1 IP and core IP are just variables to hide my physical IP addresses

17

Is prometheus running on your relay 2 node?

18

The prometheus.service is running in all three nodes

19

Prometheus only needs to be Installed on 1 server and scrape all your nodes

20

I have stopped running prometheus in the other nodes. Still no luck…