Post Upgrade to 1.25.1, some Prometheus/Grafana metrics not showing

After the upgrade (using the Guild scripts), a number of node-based metrics are no longer being reported.

As a primary example the cardano_node_BlockFetchDecision_peers_connectedPeers_int is not returning data.

I’m using Prometheus and Grafana with the standard SKY Core Node dashboard installed. The data sources are confirmed as working (many other stats showing).

Further, the gLiveView shows connected peers.

So I’m convinced everything with the relay node is fine, however am concerned that these metrics are not being shown in the monitoring tools.

I’m tipping the upgrade has changed something. I’m just not sure what, or if there is a setting / param that I am missing somewhere.

Would greatly appreciate any assistance.

from the release notes:
Stake Pool Operators (SPOs) and Exchanges should take note that the metric namespace has undergone consolidation, so all metrics now reside in cardano.node.metrics :

  • cardano.node.Forge.metrics.* → cardano.node.metrics.Forge.*
  • cardano.node.ChainDB.metrics.* → cardano.node.metrics.ChainDB.*
  • cardano.node.BlockFetchDecision.connectedPeers → cardano.node.metrics.connectedPeers

The node configs require no changes, but allow dropping entries that became redundant: wherever cardano.node.Forge.metrics.* , cardano.node.ChainDB.metrics.* or cardano.node.BlockFetchDecision.connectedPeers were mentioned, those entries can be removed. Only cardano.node.metric needs to remain. Please see #2281 for further details.

Modify ur metrics in grafana pannel like this

Old

cardano_node_BlockFetchDecision_peers_connectedPeers_int

New

cardano_node_meterics_connectedPeers_int

PS: new format: cardano_node_metrics_xxxx

Cheers,

2 Likes

Amazing and thankyou! Yes I was so close, and hadn’t seen these metrics available in the options!

Appreciate the very quick reply guys!

New

cardano_node_meterics_connectedPeers_int

Shouldn’t this be:

cardano_node_metrics_connectedPeers_int

1 Like

Hey everyone, I had the same problem and was able to get the fields fixed except 1: uptime. The key in it seems to follow the newer convention: cardano_node_metrics_upTime_ns I’ve tried a lot of different values but nothing has worked. Does anyone know what the correct key is? Also, what would I reference to get this information for the future?

I use this one, but is the uptime of vps: (time()-node_boot_time_seconds)

Hi there, I just recently updated the dashboard and for the uptime metric I use this:

Add the following metric:

  • time() - cardano_node_metrics_nodeStartTime_int

Add the following override:

  • Fields returned by query = A
  • Add override property = Unit
  • Option = duration (d hh:mm:ss)

You should see something like the following:


core-node-0
8 d 05:05:40
relay-node-0
2 d 07:26:58

Both solutions work great!!! Thank you both for your solutions. Love this dashboard