Missing categories of metrics in Prometheus

Staking & Delegation Operate a Stake Pool
21

Wow, I can’t believe I neglected to post that, sorry. 1.30.1.

22

I just downgraded my BP in testnet to same version and I still can see those metrics.

In your example I don’t see KES related metrics. Maybe you have incorrectly configured your block producer, so it do not have access to keys?

What you see when you run:

ps -ax | grep cardano-node
23

@jimerman I would like to add that if block producer do not have keys it will behave as relay. I’m thinking now that this is what happening here.

24

Ah, that may be it. I probably messed up the startup command, it wasn’t clear to me as my first node being set up.

206255 pts/1 Sl 603:54 /home/stakepool/.local/bin/cardano-node run --database-path /home/stakepool/cnode/db-block --socket-path /home/stakepool/cnode/sockets/node.socket --port 3001 --config /home/stakepool/cnode/config/mainnet-config.json --topology /home/stakepool/cnode/config/mainnet-topology.json

25

Aha, I see what I did wrong! I misread the installation instructions, and got confused as to the startup parameters. I will fix and try again. Thanks for pointing me in the right direction!

26

Please report if it was a fix for you, I’m curious to find out. :slight_smile:

if it was, please mark this as fixed. :slight_smile:

27

Yes, definitely you have incorrect command. Check this:

cat > $NODE_HOME/startBlockProducingNode.sh << EOF 
DIRECTORY=$NODE_HOME
PORT=6000
HOSTADDR=0.0.0.0
TOPOLOGY=\${DIRECTORY}/${NODE_CONFIG}-topology.json
DB_PATH=\${DIRECTORY}/db
SOCKET_PATH=\${DIRECTORY}/db/socket
CONFIG=\${DIRECTORY}/${NODE_CONFIG}-config.json
KES=\${DIRECTORY}/kes.skey
VRF=\${DIRECTORY}/vrf.skey
CERT=\${DIRECTORY}/node.cert
/usr/local/bin/cardano-node run +RTS -N -A16m -qg -qb -RTS --topology \${TOPOLOGY} --database-path \${DB_PATH} --socket-path \${SOCKET_PATH} --host-addr \${HOSTADDR} --port \${PORT} --config \${CONFIG} --shelley-kes-key \${KES} --shelley-vrf-key \${VRF} --shelley-operational-certificate \${CERT}
EOF

from here:

28

Yes, I think it worked! Thank you so much! I have learned a lot.

curl localhost:12798/metrics | grep cardano

cardano_node_metrics_slotsMissedNum_int 1999
cardano_node_metrics_served_block_count_int 345
cardano_node_metrics_submissions_submitted_count_int 34665
cardano_node_metrics_Stat_threads_int 14
cardano_node_metrics_density_real 4.939287919324964e-2
cardano_node_metrics_epoch_int 307
cardano_node_metrics_Forge_node_not_leader_int 11107
cardano_node_metrics_txsInMempool_int 33
cardano_node_metrics_delegMapSize_int 1008170
cardano_node_metrics_RTS_gcMinorNum_int 59244
cardano_node_metrics_txsProcessedNum_int 30030
cardano_node_metrics_utxoSize_int 4541797
cardano_node_metrics_RTS_gcLiveBytes_int 3517035272
cardano_node_metrics_mempoolBytes_int 25289
cardano_node_metrics_nodeStartTime_int 1639158055
cardano_node_metrics_operationalCertificateExpiryKESPeriod_int 412
cardano_node_metrics_RTS_gcMajorNum_int 869
cardano_node_metrics_blockNum_int 6611317
cardano_node_metrics_remainingKESPeriods_int 45
cardano_node_metrics_Stat_cputicks_int 264551
cardano_node_metrics_submissions_accepted_count_int 30063
cardano_node_metrics_RTS_gcticks_int 119406
cardano_node_metrics_RTS_mutticks_int 145144
cardano_node_metrics_currentKESPeriod_int 367
cardano_node_metrics_Forge_forge_about_to_lead_int 11107
cardano_node_metrics_submissions_rejected_count_int 4601
cardano_node_metrics_connectedPeers_int 2
cardano_node_metrics_served_header_counter_int 1283
cardano_node_metrics_Mem_resident_int 7417798656
cardano_node_metrics_operationalCertificateStartKESPeriod_int 350
cardano_node_metrics_slotInEpoch_int 344919
cardano_node_metrics_slotNum_int 47605719
cardano_node_metrics_RTS_gcHeapBytes_int 10488905728
1 Like
29

FYI, I run cardano-node as a service. I create /etc/systemd/system/cardano-node.service:

[Unit]
Description=PLL Cardano Block Producer
After=multi-user.target
[Service]
Type=simple
ExecStart=/home/stakepool/.local/bin/cardano-node run \
    --config /home/stakepool/cnode/config/mainnet-config.json \
    --topology /home/stakepool/cnode/config/mainnet-topology.json \
    --database-path  /home/stakepool/cnode/db-block/  \
    --socket-path  /home/stakepool/cnode/sockets/node.socket \
    --host-addr 0.0.0.0 \
    --port 3001 \
    --shelley-kes-key /home/stakepool/cnode/keys/myPool.kes-000.skey \
    --shelley-vrf-key /home/stakepool/cnode/keys/myPool.vrf.skey \
    --shelley-operational-certificate /home/stakepool/cnode/keys/myPool.node-000.opcert

KillSignal = SIGINT
RestartKillSignal = SIGINT
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=cardano
LimitNOFILE=32768

Restart=on-failure
RestartSec=15s
WorkingDirectory=~
User=stakepool
Group=stakepool
[Install]
WantedBy=multi-user.target

Then I register and start it:

sudo systemctl enable cardano-node
sudo systemctl start cardano-node
30

Hi @jimerman

Please mark this is as fixed.