Cardano node was not cleanly shutdown

Hello,

I’m working on getting my second node running, but I’m having issues syncing with the network.
I think the problem was that I tried to run the node for the first time in tmux, it frozen so I closed it and now I get the following error message:
WARN: A prior running Cardano node was not cleanly shutdown, socket file still exists. Cleaning up.
Listening on http://127.0.0.1:12798
I tried leaving it like this for a while to see if it fixed itself but nothing happen. Suggestions?

Hi Martin,

Maybe try stopping your node manually and then executing killall cardano-node

If that doesn’t work, try removing your node.socket file and restarting the node

If all else fails you can wipe the entire database directory

Your friend, FROG

Hi Frog,

Thanks for your response. I tried killall cardano-node but got no process found. So I went and removed socket0.node. I was now able to run it without getting the warning message but I only got Listening on http://127.0.0.1:12798 as message and nothing more. The actual node is not opening…
What should I do? :disappointed_relieved:

try

rm -R /path/to/cardano-node/db

Your friend, FROG

I tried removing db directory and node.stocket still same outcome, it gets stuck at listening…
any other idea? :sweat_smile:

try using 0.0.0.0 as the host-addr param when starting the node

where do I find that parameter? in the config file?
thanks for the help! :slight_smile:

My pleasure

It will either be in whatever script you are using to start the node, or used as a parameter when invoking that script

so I checked the script and the host address was okay, I did change the port which I had changed in the topology file and it was not updated in the script. But it’s still stuck in listening…

you can always check if you have any open tmux session by typing:

tmux list-sessions

If there is one (or more) you can attach to latest session by typing

tmux a

if you have any frozen tmux session - you can kill it with:

tmux kill-server

before that, try to gracefully kill the cardano-node processes with:
killall -s SIGKILL cardano-node

if you are using CNtools scripts to run your nodes, after forced cardano-node shutdown (before it closes db and removes socket file) in some cases you will have to manually remove the socket file and only then start the script

Do you run your second node on the same server? If yes, then don’t forget to change in start script the db directory and socket file names so they are different from each node. It will not work if you are pointing to the same socket / db dir.

1 Like

Hi lauris thanks for your feedback and sorry for the late reply. I still haven’t been able to solve this issue.
I don’t have any tmux session running, using the killall command returns “no process found” and I tried several times already deleting the socket file and db but still get the same error message…
What else could I do? delete everything and start from scratch?

can you share how are you starting the node?
what’s in the startup script?

Hi lauris, I’m using the scripts from the guild operators page.
I’m using ./cnode.sh which contains the following:

#!/bin/bash
# shellcheck disable=SC2086

[[ -z "$CNODE_HOME" ]] && CNODE_HOME=/opt/cardano/cnode

. "${CNODE_HOME}"/scripts/env

if [[ -S "$CNODE_HOME/sockets/node0.socket" ]]; then
  if [ `ps -ef | grep -c [c]ardano-node.*.$CNODE_HOME/sockets/node0.socket` -gt 0 ]; then
     echo "ERROR: A Cardano node is already running, please terminate this node before starting a new one with this script."
     exit 1
  else
    echo "WARN: A prior running Cardano node was not cleanly shutdown, socket file still exists. Cleaning up."
    unlink $CNODE_HOME/sockets/node0.socket
  fi
fi

[[ ! -d "$CNODE_HOME/logs/archive" ]] && mkdir -p "$CNODE_HOME/logs/archive"

[[ $(find "$CNODE_HOME"/logs/*.json | wc -l) -gt 0 ]] && mv $CNODE_HOME/logs/*.json $CNODE_HOME/logs/archive/

cardano-node run \
        --topology $CNODE_HOME/files/topology.json \
        --config $CNODE_HOME/files/config.json \
        --database-path $CNODE_HOME/db \
        --socket-path $CNODE_HOME/sockets/node0.socket \
        --host-addr 0.0.0.0 \
        --port 6003
"$POOL_DIR/hot.skey" ]]; then
  cardano-node run \
        --topology $CNODE_HOME/files/topology.json \
        --config $CNODE_HOME/files/config.json \
        --database-path $CNODE_HOME/db \
        --socket-path $CNODE_HOME/sockets/node0.socket \
        --host-addr 0.0.0.0 \
        --shelley-kes-key $POOL_DIR/hot.skey \
        --shelley-vrf-key $POOL_DIR/vrf.skey \
        --shelley-operational-certificate $POOL_DIR/op.cert \
        --port $CNODE_PORT
else
  cardano-node run \
        --topology $CNODE_HOME/files/topology.json \
        --config $CNODE_HOME/files/config.json \
        --database-path $CNODE_HOME/db \
        --socket-path $CNODE_HOME/sockets/node0.socket \
        --host-addr 0.0.0.0 \
        --port $CNODE_PORT
fi

## End

Okay, so I run prereqs.sh again to update the files. I updated the port and the topology file. I run the cnode.sh again and is still stuck on Listening on http://127.0.0.1:12798 :sob:

do you have any log file? maybe in logs folder?
as it looks like it starts, but doesn’t report any movement to the console.
check config file for:
“defaultScribes”: [
[
“StdoutSK”,
“stdout”
]
],

or maybe MinSeverity settings are too high.
“minSeverity”: “Debug”,

which version of cardano-node you have?

Hi lauris, thanks again for your feedback. I have node version 1.19.0
I do have logs files in the logs folder, every single one has the following information

{“at”:“2020-09-21T18:22:45.68Z”,“env”:“1.19.0:48140”,“ns”:[“cardano.node.ChainSyncClient”],“data”:{“kind”:“ChainSyncClientEvent.TraceDownloadedHeader”,“block”:{“kind”:“BlockPoint”,>

In the config file I have the following

 "defaultScribes": [
    [
      "FileSK",
      "/opt/cardano/cnode/logs/node0.json"
    ]
  ],

and
"minSeverity": "Info",

i would suggest to reinstall from scratch, here is an unofficial guide i have created, hope this helps:

Thanks lauris. What’s the easiest way to uninstall everything and start again?

it depends where you store your scripts/logs/db, etc… if not in the cardano-node source folder, then just delete (or just in case, rename) and that’s it.

Hi @lauris
I have the same issue, getting Listening on http://127.0.0.1:12798 when running ./cnode.sh script
I build up my servers from scratch twice but still the same.
I appreciate your resolution steps in advance