Cardano node gets stuck after a while

Developers Misc Dev Talk
1

Hi there.

I’m running my own Cardano full node (latest v1.31), but every time when it gets started for a while, it suddenly closes its connection and gets stuck.

I can only see the logs like below from the node now. It’s slowly but surely printing out a higher number of slot numbers constantly.

(...)
[<NAME>:cardano.node.ChainDB:Notice:35] [2021-11-26 11:56:49.00 UTC] Chain extended, new tip: c86804974b33494ecca57d874a3fcc60b9c777b66cb4960ac3331dd04b8246b2 at slot 39922586
[<NAME>:cardano.node.ChainDB:Notice:35] [2021-11-26 11:56:50.33 UTC] Chain extended, new tip: 38a54f44431715b52206fab30d129e6f22b3a943a2647cfa90721d5edd4fabef at slot 39923265
[<NAME>:cardano.node.ChainDB:Notice:35] [2021-11-26 11:56:51.58 UTC] Chain extended, new tip: 6fc869151a0f6b233f90eb2dcd24941328490743580e736954f8cbf8591df293 at slot 39924056

Based on my understanding, slot number 39924056 is quite outdated from the latest one.

Does it mean that my node is still having a sync issue or something?

It used to respond via node.socket for a while after I initially started, but all of sudden it stopped responding through the socket and fall into this situation.

Any idea what I’m experiencing?

Thank you.

2

Hi,

What is the server HW configuration?
can u type free -m and share the output?

3

Hi.

I’m running it on the 4vCore 8GB virtual machine (with NVMe SSD), on the DigitalOcean Cloud.

Here’s the result of the free -m command.

root@<HOSTNAME>:~/cardano-node-1.31.0# free -m
              total        used        free      shared  buff/cache   available
Mem:           7961         334        6308           1        1318        7369
Swap:             0           0           0

And the result of df -h command for just in case

root@<HOSTNAME>:~/cardano-node-1.31.0# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.9G     0  3.9G   0% /dev
tmpfs           797M  1.1M  796M   1% /run
/dev/vda1       155G   80G   76G  52% / *### The one I'm using for Cardano Node ###*
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda15      105M  5.2M  100M   5% /boot/efi
/dev/loop2       68M   68M     0 100% /snap/lxd/21835
/dev/loop3       33M   33M     0 100% /snap/snapd/13640
/dev/loop4       68M   68M     0 100% /snap/lxd/21803
/dev/loop6       62M   62M     0 100% /snap/core20/1242
/dev/loop0       43M   43M     0 100% /snap/snapd/14066
overlay         155G   80G   76G  52% /var/lib/docker/overlay2/5b53c70dc8b62f70e50e648c1d75b662b95a87faac183c5128cd4265481a3d7d/merged
shm              64M     0   64M   0% /var/lib/docker/containers/04cce30c113c6fe3694fc3b9233418074bfaecbc0258c8e00637b0200e29698d/mounts/shm
/dev/loop5       62M   62M     0 100% /snap/core20/1270
tmpfs           797M     0  797M   0% /run/user/0

Thanks.

4

ok, 8G is not enough for 1.31.0 so I will recommend to configure the SWAP file (at least 4G but you can use 6G)

the commands are:

sudo fallocate -l 6G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

check again with free -m (now you should see 6G allocated for SWAP file)

to make this setting permanent (not deleted when the machine will be rebooted) type

sudo nano /etc/fstab

and add this line to the end and save the file

/swapfile swap swap defaults 0 0

now for testing you can restart the machine

keep it under observation for a while

1 Like
5

Hey there.

Thanks for your answer. I increased the physical memory to 16GB, and the node started to function.

Thank you so much!

1 Like
6

Even better, you are welcome!

1 Like