I ran the script 18. Operational and Maintenance Tips - CoinCashew under the heading Running LeaderLog with stake-snapshot.
It spent a long time synching then failed with an error. Then my block producer node stopped listening on port 6000 and is spiked at 100% cpu utilization. Even after a reboot the system is no longer listening on port 6000 and not communicating with my relay node. Why does this happen? Is the code that unstable?
What is the hardware configuration of the node? Perhaps crashed due to not enough memory?
It is a cloud server with 6 CPUS and 16 gig of ram. Top says that cpu is spiked but memory shows 5812.1 MB free. And top reads the following:
MiB Mem: 15988.8 total 191.0 free 5850.5 used 9948 buff/cache
MiB Swap: 512.0 total, 512.0 free, 0.0 used 9854.6 avail Mem
Cardono node is using 1025.7 g VIRT, 5.6G RES and 35.6 mem, but has calmed down to 35.6 %cpu
It was running fine till I ran that script and then all heck broke loose, even after a reboot and a restart of cardano-node the CPU stayed spiked for a while before surfacing port 6000
Aaa yes, every time when the node is restarting u will see the cpu ~100% til the node will start/sync
It is normal.
Now check with journalctl -e -f -u cardano-node if u see the killed message (when it crashed )
If yes then u need to add SWAP file in case u don’t have it or increase it to 4-6G
Running this script definately causes cardano-node to at least double it’s memory consumption:
/usr/local/bin/cncli sync --host 127.0.0.1 --port 6000 --no-service
MYPOOLID=$(cat $NODE_HOME/stakepoolid.txt)
echo “LeaderLog - POOLID $MYPOOLID”
SNAPSHOT=$(/usr/local/bin/cardano-cli query stake-snapshot --stake-pool-id $MYPOOLID --mainnet)
POOL_STAKE=$(echo “$SNAPSHOT” | grep -oP ‘(?<= “poolStakeMark”: )\d+(?=,?)’)
ACTIVE_STAKE=$(echo “$SNAPSHOT” | grep -oP ‘(?<= “activeStakeMark”: )\d+(?=,?)’)
MYPOOL=/usr/local/bin/cncli leaderlog --pool-id $MYPOOLID --pool-vrf-skey ${NODE_HOME}/vrf.skey --byron-genesis ${NODE_HOME}/mainnet-byron-genesis.json --shelley-genesis ${NODE_HOME}/mainnet-shelley-genesis.json --pool-stake $POOL_STAKE --active-stake $ACTIVE_STAKE --ledger-set next
echo $MYPOOL | jq .
EPOCH=echo $MYPOOL | jq .epoch
echo “`Epoch $EPOCH` 
:”
SLOTS=echo $MYPOOL | jq .epochSlots
IDEAL=echo $MYPOOL | jq .epochSlotsIdeal
PERFORMANCE=echo $MYPOOL | jq .maxPerformance
echo “`MYPOOL - $SLOTS `:slot_machine:`, $PERFORMANCE% `:four_leaf_clover:max, `$IDEAL` 
ideal”
free -m
What size has the SWAP file? I know that when I run leaderlog script it will use aditional ~8G of RAM
So it was 512 MB but I just bumped it up to 7680 MB and I am restarting the server.
Ok ran the script again and didn’t tip over the server. Virtual mem is absolutely needed. But the script says "status’: “error”, “errorMessage”: "Not enough blocks sync’d to calculate! Try again later after slot 46699200 is sync’d.
Early in the log we get an entry:
WARN cardano_oroboros_network::protocols::chainsync > rollback to slot 38675707
you must sync it first…