Cnode service reboots every 10 min on BP after 1.27.0 upgrade

Do u have the possibility to update one node for test?

Do you mean more resources ?

Or type top and try to monitor the cardano-node process… check if the mem will go to 100%

Tihs exit code usually means that the process was eating up the memory…
yes, good to monitor the cardano-node process and examine the memory allocation:
top -p <pid> get the resource usage by process id

1 Like

May 24 10:24:08 SPCORENODE01 systemd[1]: cnode.service: Main process exited, code=exited , status=137/n/a

137 is a resource issue error code. Ensure you have at least 8gb of memory and enough cpu computing power to launch the node. Potentially add some 16gb of swap as well. Also Are you fully synced? And you do not have multiple nodes running on the same server/instance.

Looks like cncli constantly running as service could you please advice how to remove it from service.

is this the producer?

Run ./deploy-as-systemd.sh

Preas N for topology updater and N for the rest (CNCLI)

Yes, this is producer. Thank you guys for your help

Hi guys, conclusion to wrap up case.
Alex was right from the start, lack of resources caused the issue. Not sure why it was working for 3 days after update to 1.27.
Thank to all of you guys trying to help Alex, laplasz and ToTheMoonADA.
Sad to admit but I missed my first block ever because of this :frowning:
2021-05-24 13_09_03-KAAdmin@SPCORENODE01_ _opt_cardano_cnode_scripts

1 Like

Yes, but that should be communicated back to cardano-node developers - that 1.27.0 need more memory then before… I thought 8 GB would be enough… still on 1.26.2 but I wont update till there will be a solution for this…
I recommend to submit an issue on github:

U didn’t said u have a block :frowning:

2 Likes

Man It would not change anything, that happened in the night. I Started topic after the fact.

Will let them know.

thx - if you have the github link about the report please let us know…

ohhh you got a block!! congrats.
Guess that answers the fully sync question =).

1.27 slowly eats up resources.

1 Like

@laplasz Problem is that I had 2 vCPU and 8 GB so I was sure I am running 4vCPU I am not sure if that’s going to be appropriate to report, looks like I am massed up.

@ToTheMoonADA , thanks man I was happy too before I missed it :wink:

Hi Anton, what’s the final vcpu/ram/swap setting you use now to have the issue resolve?

Hi Tony, BP - 4vCPU 16 GB RAM no SWAP (do not ask about 16 :slight_smile: This is just next step in Azure)
Relays - 2 vCPU 8 GB RAM each no SWAP

U can fix the MEM on relays by going inside the config file and set the TraceMempool=false
And restart the node

It will reduce the MEM on relays… u will not see TX processed anymore on relays but u will see on BP

2 Likes