1.34.1 Guild LiveView terminated, cleaning up

I’ve been trying to set up a pool with cardano-node 1.34.1

However, the nodes sync up but crash with error:

Guild LiveView terminated, cleaning up…

Is it the current release of cardano-node? nodes did not do this on 1.33

1 Like

journalctl -e -f -u cardano-node and check if u see any errors or kill messages
also check the RAM with free -m or the disk space available df -h

cheers,

1 Like

Hi,

with journalctl i am getting:

Mar 30 12:21:10 node systemd[1]: cardano-node.service: Failed with result ‘exit-code’.

with free -m i am getting:

    total used free shared buff/cache  available

Mem: 16000 144 15411 0 445 15604
Swap: 8191 0 8191

and with df -h i got:

Filesystem Size Used Avail Use% Mounted on
/dev/root 68G 67G 1.1G 99% /
devtmpfs 7.9G 0 7.9G 0% /dev
tmpfs 7.9G 0 7.9G 0% /dev/shm
tmpfs 1.6G 904K 1.6G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/loop0 56M 56M 0 100% /snap/core18/2344
/dev/loop1 265M 265M 0 100% /snap/google-cloud-sdk/231
/dev/loop2 44M 44M 0 100% /snap/snapd/15177
/dev/loop3 292M 292M 0 100% /snap/google-cloud-sdk/232
/dev/sda15 105M 5.2M 100M 5% /boot/efi
tmpfs 1.6G 0 1.6G 0% /run/user/1001

I do not understand. 1.33 was running fine for weeks.

Thank you @Alexd1985

It looks like you have used 99% of your root filesystem space. If I am reading that correctly, that is not good.

Which filesystem is your cardano-node storing its blockchain data on?

It is a 70 GB instance. Used to work fine with 60 GB and 12 GB of RAM with cardano-node 1.33
Now I am trying with 1.34.1 70 GB and 16 of RAM and having trouble after they synced to 99%

This time I added 8 GB swap. I don’t know if that affects the memory allocation of the node db

You are going to need a lot more space than 70G. The blockchain data alone is currently 48G. If you also run cncli-sync which stores its own sqlite database then that will use another 4G currently. And all this data continues to grow. You still need space for the operating system files. My basic Debian system uses 11G for the operating system files. Also, depending on which filesystem you use there needs to be space for its journalling records etc. You can’t fill your filesystem to 100%. Going past 80% is often a problem.

I would set aside 200G as this would give you enough spare space for the blockchain data to grow for some time. Recall that IOG is currently changing parameters to increase blockchain throughput. This means more data storage.

I would also recommend 16G of RAM and 16G of swap space.

Thank you. I will definitely try that

1 Like

as far as I know the db has ~52G right now… perhaps you will need to delege the logs from var/log/journal or something like this

I tried deleting that folder and I still get “connection to node lost, retrying…”
I will add more storage and swap and see if that helps. Thank you @Alexd1985

1 Like

Hi @Alexd1985

I started from scratch with a new instance with 16 GB of RAM, 4 cores and 200 GB of storage and 16GB of swap.

The relay node is synced and working fine, but the block producer:

“Connection to node lost, retrying”

after I added kes.skey, node.cert and vrf.skey.

The pool has been running since january and I had rotated the KES once. So I added to the node:

  • vrf.skey (original)
  • node.cert (from rotation)
  • kes.skey (from rotation)

Am I doing something wrong rebuilding the block producer? It has the right topology.

Thank you.

perhaps the permision for vrf.skey file is wrong (should be read only)… type journalctl -e -f -u cardano-node or cnode and check for errors

2 Likes

That was exactly it. Thank you so much. Have a good day.

2 Likes

you’re welcome. :beers:

hi, I have similar problem with ./gLiveView.sh (LiveView terminated, cleaning up…)
free -m( total used free shared buff/cache available
Mem: 15914 8832 1012 79 6070 6675
Swap: 2047 1 2046)

df -h (Filesystem Size Used Avail Use% Mounted on
tmpfs 1.6G 1.8M 1.6G 1% /run
/dev/sda2 457G 42G 393G 10% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
/dev/sda1 511M 5.3M 506M 2% /boot/efi
tmpfs 1.6G 2.4M 1.6G 1% /run/user/1000)

cnode 1.35.3 (synched)

journalctl -e -f -u cardano-node(shows no errors)

I have ubuntu, 2 core intel, 16 ram, 500MB on Relay

and also I cant set up grafana, but this is discussion for later

journalctl -e -f -u cnode (or cardano-node) and check for errors

○ → journalctl -e -f -u cardano-node
Oct 01 18:34:42 ada cardano-node[974]: [ada:cardano.node.ChainDB:Notice:169] [2022-10-01 17:34:42.03 UTC] Chain extended, new tip: f048ddccace204ff7b9bcfb853df8a72d722da467686d246070e1e29dbb56ed3 at slot 48792869
Oct 01 18:34:42 ada cardano-node[974]: [ada:cardano.node.ChainDB:Info:169] [2022-10-01 17:34:42.09 UTC] Valid candidate f5e16c50afde956828e20af7e87c60793c17ea67f6868db7080e871e613f0c77 at slot 48792921

shows no errors

Then check inside env file which cnode port did u have set (should be the same port as the port used when u started the node… and uncomment the line)

env file is also ok. yesterday I managed to set up all and ./gLiveView.sh worked ( it was synching over night). when I woke up I saw that IP changed and set it back on previous IP that I set forwarding rules to and this problem started

1 Like