Failed to load common env after updating topology.json

This block producer has been running for months. I upgraded it to 1.35.3 without issue and changed its topology.json a couple of times. However, after adding a new relay to its topology.json it no longer wants to start.

Sudo systemctl status cnode returns the following:

Loaded: loaded (/etc/systemd/system/cnode.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2022-09-09 17:13:53 UTC; 602ms ago
Process: 5052 ExecStart=/bin/bash -l -c exec /opt/cardano/cnode/scripts/cnode.sh (code=exited, status=1/FAILURE)
Main PID: 5052 (code=exited, status=1/FAILURE)

journalctl -e -f -u cnode.service returns the following:

ERROR: Failed to load common env file

The env file has not been adjusted. All lines are commented except for CNODE_PORT and POOL_NAME.

I checked the topology.json for errors but couldn’t find them. All entries follow this format:

{
"Producers":
[
{
"addr": "X",
"port": X,
"valency": 1
},
{
"addr": "X",
"port": X,
"valency": 1
},
{
"addr": "X",
"port": X,
"valency": 1
},
{
"addr": "X",
"port": X,
"valency": 1
}
]
}

I tried looking for a solution on this forum but couldn’t find one that works for me. Perhaps someone has an idea of what might be wrong here? Help is much appreciated!

Check your topology again, really carefully. I had a problem like that and I had just missed a , or something similar. I had overlooked it many times before I spotted it.

1 Like

I checked it numerous times and rewrote all of it multiple times. Still no go sadly.

type

journalctl -e -f -u cnode 

and check for errors (check the logs since last restart)… u must have more lines that the above one shared… if not stop the node

sudo systemctl stop cnode

and try to start it manually from scripts folder ./cnode.sh and check the message

I stopped cnode and attempted a manual restart. I still get the following message in the journal:

ERROR: Failed to load common env file
Please verify set values in 'User Variables' section in env file or log an issue on GitHub

I guess I can uncomment the lines in env once more.

wait… before to uncomment check with journalctl
check carefully… u should see more errors… after u ran ./cnode.sh what did u see in terminal?

When I start ./cnode.sh I see the node configuration and it ends with

DbLocked "/opt/cardano/cnode/db/lock"

The journalctl -e -f -u cnode still states Error: Failed to load common env file. No other codes. I see that gLiveView is working again but the Block, Slot and Slot epoch just show 0.

Try rebooting the server.

I rebooted the server multiple times in the last hour. Perhaps I should allow it more time?

The node keeps wanting to start but failing in doing so. At this point I am considering just creating a new node. Which would be weird since the node software hasn’t been touched. I just edited the topology.json file…

I would try putting in the last working content of the topology.json and see if that works.

hmm
Try sudo systemctl status cnode
and sudo systemctl status cardano-node

coincashew or cntools guide?

I use cntools and the cnode command presents the following:

cnode.service - Cardano Node
     Loaded: loaded (/etc/systemd/system/cnode.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2022-09-09 23:10:09 UTC; 4s ago
    Process: 9447 ExecStart=/bin/bash -l -c exec /opt/cardano/cnode/scripts/cnode.sh (code=exited, status=1/FAILURE)
   Main PID: 9447 (code=exited, status=1/FAILURE)

I just redid the prereqs and set the env. Same error as before.

try

cd ~/tmp
./prereqs.sh -f

It will download the default files again
Then restart the node and check if it will start … if yea stop it and edit the files (config.json, topology.json, cnode.sh if u edited before, env file, etc)

I redownloaded the files but the node still doesn’t start (keeps restarting). I still get the message

ERROR: Failed to load common env

I am kind of at a loss here

type top and check if u see cardano-node running and check with journalctl … u must have more error messages than failed to load env

The CPU is not doing much and journalctl keeps returning

Error: Failed to load common env
Please verify set values in 'User Variables' section in env file or log an issue on Github

When I attempt to start gLiveView I get something similar

Node socket not set in env file and automatic detection failed! [source: gLiveView.sh]

ERROR: gLiveView failed to load common env file
Please verify set values in 'User Variables' section in env file or log an issue on GitHub

then u have more issues there
try sudo systemctl stop cnode
sudo systemctl disable cnode
and try to connect the node manually… the db is still locked?
type sudo systemctl and check what processes are running

I stopped cnode, disabled cnode, rebooted the server and then executed ./cnode.sh. This is what I see:

Node configuration: NodeConfiguration {ncSocketConfig = SocketConfig {ncNodeIPv4Addr = Last {getLast = Just 0.0.0.0}, ncNodeIPv6Addr = Last {getLast = Nothing}, ncNodePortNumber = Last {getLast = Just 6001}, ncSocketPath = Last {getLast = Just “/opt/cardano/cnode/sockets/node0.socket”}}, ncConfigFile = “/opt/cardano/cnode/files/config.json”, ncTopologyFile = “/opt/cardano/cnode/files/topology.json”, ncDatabaseFile = “/opt/cardano/cnode/db”, ncProtocolFiles = ProtocolFilepaths {byronCertFile = Nothing, byronKeyFile = Nothing, shelleyKESFile = Just “/opt/cardano/cnode/priv/pool/Ideal/hot.skey”, shelleyVRFFile = Just “/opt/cardano/cnode/priv/pool/Ideal/vrf.skey”, shelleyCertFile = Just “/opt/cardano/cnode/priv/pool/Ideal/op.cert”, shelleyBulkCredsFile = Nothing}, ncValidateDB = False, ncShutdownConfig = ShutdownConfig {scIPC = Nothing, scOnSyncLimit = Just NoShutdown}, ncProtocolConfig = NodeProtocolConfigurationCardano (NodeByronProtocolConfiguration {npcByronGenesisFile = “/opt/cardano/cnode/files/byron-genesis.json”, npcByronGenesisFileHash = Nothing, npcByronReqNetworkMagic = RequiresNoMagic, npcByronPbftSignatureThresh = Nothing, npcByronApplicationName = ApplicationName {unApplicationName = “cardano-sl”}, npcByronApplicationVersion = 1, npcByronSupportedProtocolVersionMajor = 3, npcByronSupportedProtocolVersionMinor = 0, npcByronSupportedProtocolVersionAlt = 0}) (NodeShelleyProtocolConfiguration {npcShelleyGenesisFile = “/opt/cardano/cnode/files/shelley-genesis.json”, npcShelleyGenesisFileHash = Nothing}) (NodeAlonzoProtocolConfiguration {npcAlonzoGenesisFile = “/opt/cardano/cnode/files/alonzo-genesis.json”, npcAlonzoGenesisFileHash = Nothing}) (NodeHardForkProtocolConfiguration {npcTestEnableDevelopmentHardForkEras = False, npcTestShelleyHardForkAtEpoch = Nothing, npcTestShelleyHardForkAtVersion = Nothing, npcTestAllegraHardForkAtEpoch = Nothing, npcTestAllegraHardForkAtVersion = Nothing, npcTestMaryHardForkAtEpoch = Nothing, npcTestMaryHardForkAtVersion = Nothing, npcTestAlonzoHardForkAtEpoch = Nothing, npcTestAlonzoHardForkAtVersion = Nothing, npcTestBabbageHardForkAtEpoch = Nothing, npcTestBabbageHardForkAtVersion = Nothing}), ncDiffusionMode = InitiatorAndResponderDiffusionMode, ncSnapshotInterval = DefaultSnapshotInterval, ncTestEnableDevelopmentNetworkProtocols = False, ncMaxConcurrencyBulkSync = Nothing, ncMaxConcurrencyDeadline = Just 2, ncLoggingSwitch = True, ncLogMetrics = True, ncTraceConfig = TracingOnLegacy (TraceSelection {traceVerbosity = NormalVerbosity, traceAcceptPolicy = OnOff {isOn = False}, traceBlockFetchClient = OnOff {isOn = True}, traceBlockFetchDecisions = OnOff {isOn = True}, traceBlockFetchProtocol = OnOff {isOn = False}, traceBlockFetchProtocolSerialised = OnOff {isOn = True}, traceBlockFetchServer = OnOff {isOn = False}, traceBlockchainTime = OnOff {isOn = False}, traceChainDB = OnOff {isOn = True}, traceChainSyncBlockServer = OnOff {isOn = True}, traceChainSyncClient = OnOff {isOn = False}, traceChainSyncHeaderServer = OnOff {isOn = False}, traceChainSyncProtocol = OnOff {isOn = False}, traceConnectionManager = OnOff {isOn = True}, traceConnectionManagerCounters = OnOff {isOn = True}, traceConnectionManagerTransitions = OnOff {isOn = False}, traceDebugPeerSelectionInitiatorTracer = OnOff {isOn = False}, traceDebugPeerSelectionInitiatorResponderTracer = OnOff {isOn = False}, traceDiffusionInitialization = OnOff {isOn = False}, traceDnsResolver = OnOff {isOn = False}, traceDnsSubscription = OnOff {isOn = True}, traceErrorPolicy = OnOff {isOn = True}, traceForge = OnOff {isOn = True}, traceForgeStateInfo = OnOff {isOn = True}, traceHandshake = OnOff {isOn = False}, traceInboundGovernor = OnOff {isOn = False}, traceInboundGovernorCounters = OnOff {isOn = True}, traceInboundGovernorTransitions = OnOff {isOn = True}, traceIpSubscription = OnOff {isOn = False}, traceKeepAliveClient = OnOff {isOn = False}, traceLedgerPeers = OnOff {isOn = False}, traceLocalChainSyncProtocol = OnOff {isOn = False}, traceLocalConnectionManager = OnOff {isOn = False}, traceLocalErrorPolicy = OnOff {isOn = True}, traceLocalHandshake = OnOff {isOn = False}, traceLocalInboundGovernor = OnOff {isOn = False}, traceLocalMux = OnOff {isOn = False}, traceLocalRootPeers = OnOff {isOn = False}, traceLocalServer = OnOff {isOn = False}, traceLocalStateQueryProtocol = OnOff {isOn = False}, traceLocalTxMonitorProtocol = OnOff {isOn = False}, traceLocalTxSubmissionProtocol = OnOff {isOn = False}, traceLocalTxSubmissionServer = OnOff {isOn = False}, traceMempool = OnOff {isOn = False}, traceMux = OnOff {isOn = False}, tracePeerSelection = OnOff {isOn = False}, tracePeerSelectionCounters = OnOff {isOn = True}, tracePeerSelectionActions = OnOff {isOn = False}, tracePublicRootPeers = OnOff {isOn = False}, traceServer = OnOff {isOn = False}, traceTxInbound = OnOff {isOn = False}, traceTxOutbound = OnOff {isOn = False}, traceTxSubmissionProtocol = OnOff {isOn = False}, traceTxSubmission2Protocol = OnOff {isOn = False}}), ncTraceForwardSocket = Nothing, ncMaybeMempoolCapacityOverride = Nothing, ncProtocolIdleTimeout = 5s, ncTimeWaitTimeout = 60s, ncAcceptedConnectionsLimit = AcceptedConnectionsLimit {acceptedConnectionsHardLimit = 512, acceptedConnectionsSoftLimit = 384, acceptedConnectionsDelay = 5s}, ncTargetNumberOfRootPeers = 100, ncTargetNumberOfKnownPeers = 100, ncTargetNumberOfEstablishedPeers = 50, ncTargetNumberOfActivePeers = 20, ncEnableP2P = DisabledP2PMode}
Listening on http://127.0.0.1:12798

From this point out the terminal doesn’t change. Perhaps a noob question but how can I journalctl without interfering with this process?

Apologies for taking up so much time. The help is much appreciated.

Edit: I don’t know how a simple edit to the topology.json resulted in all of this. I ran the software_upgrade.sh script and forced a 1.35.3 upgrade just to be sure all was up to date. I edited and compared the env file, the topology.json, cnode.sh, the firewall, etc with other working nodes of mine. I don’t get it. I am creating a new node as a back-up but want to understand what it is that keeps this one from working.

this is fine, open on another terminal glive and check

You said you’ve added new relay to topology.json, which caused the problem. Then does it work again if you remove that new lines you have added.to topology.json?

1 Like