Node not starting after syncing

Hello Everyone,

My note just finished syncing last night after many issues lol. however, it is now not starting and presenting this in the logs.

  • Logs begin at Fri 2021-05-21 21:10:55 UTC. –
    May 23 17:02:22 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:22.21 UTC] Replayed block: slot SlotNo 27219544 of At (SlotNo 27263096)
    May 23 17:02:23 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:23.63 UTC] before next, messages elided = 27219600
    May 23 17:02:23 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:23.63 UTC] Replayed block: slot SlotNo 27237572 of At (SlotNo 27263096)
    May 23 17:02:23 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:23.63 UTC] Replayed block: slot SlotNo 27237591 of At (SlotNo 27263096)
    May 23 17:02:25 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:25.04 UTC] before next, messages elided = 27237609
    May 23 17:02:25 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:25.04 UTC] Replayed block: slot SlotNo 27259171 of At (SlotNo 27263096)
    May 23 17:02:25 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:25.04 UTC] Replayed block: slot SlotNo 27259176 of At (SlotNo 27263096)
    May 23 17:02:25 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:25.28 UTC] before next, messages elided = 27259201
    May 23 17:02:25 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:25.28 UTC] Replayed block: slot SlotNo 27263096 of At (SlotNo 27263096)
    May 23 17:02:25 CarnitasRelay1 cardano-node[3045]: [Carnitas:cardano.node.ChainDB:Info:5] [2021-05-23 17:02:25.28 UTC] Opened lgr db
    May 23 17:03:09 CarnitasRelay1 cardano-node[3044]: /home/tacos/cardano-my-node/startRelayNode1.sh: line 9: 3045 Killed /usr/local/bin/cardano-node run --topology ${TOPOLOGY} --database-path ${DB_PATH} --socket-path ${SOCKET_PATH} --host-addr ${HOSTADDR} --port ${PORT} --config ${CONFIG}
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: Listening on http://127.0.0.1:12798
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.networkMagic:Notice:5] [2021-05-23 17:03:16.23 UTC] NetworkMagic 764824073
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.protocol:Notice:5] [2021-05-23 17:03:16.23 UTC] Byron; Shelley
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.version:Notice:5] [2021-05-23 17:03:16.23 UTC] 1.27.0
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.commit:Notice:5] [2021-05-23 17:03:16.23 UTC] 8fe46140a52810b6ca456be01d652ca08fe730bf
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.nodeStartTime:Notice:5] [2021-05-23 17:03:16.23 UTC] 2021-05-23 17:03:16.231267642 UTC
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.systemStartTime:Notice:5] [2021-05-23 17:03:16.23 UTC] 2017-09-23 21:44:51 UTC
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotLengthByron:Notice:5] [2021-05-23 17:03:16.23 UTC] 20s
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.epochLengthByron:Notice:5] [2021-05-23 17:03:16.23 UTC] 21600
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotLengthShelley:Notice:5] [2021-05-23 17:03:16.23 UTC] 1s
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.epochLengthShelley:Notice:5] [2021-05-23 17:03:16.23 UTC] 432000
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotsPerKESPeriodShelley:Notice:5] [2021-05-23 17:03:16.23 UTC] 129600
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotLengthAllegra:Notice:5] [2021-05-23 17:03:16.23 UTC] 1s
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.epochLengthAllegra:Notice:5] [2021-05-23 17:03:16.23 UTC] 432000
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotsPerKESPeriodAllegra:Notice:5] [2021-05-23 17:03:16.23 UTC] 129600
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotLengthMary:Notice:5] [2021-05-23 17:03:16.23 UTC] 1s
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.epochLengthMary:Notice:5] [2021-05-23 17:03:16.23 UTC] 432000
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.basicInfo.slotsPerKESPeriodMary:Notice:5] [2021-05-23 17:03:16.23 UTC] 129600
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.addresses:Notice:5] [2021-05-23 17:03:16.23 UTC] [SocketInfo 0.0.0.0:6000]
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.diffusion-mode:Notice:5] [2021-05-23 17:03:16.23 UTC] InitiatorAndResponderDiffusionMode
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.dns-producers:Notice:5] [2021-05-23 17:03:16.23 UTC] [DnsSubscriptionTarget {dstDomain = “relays-new.cardano-mainnet.iohk.io”, dstPort = 3001, dstValency = 2}]
    May 23 17:03:16 CarnitasRelay1 cardano-node[3651]: [Carnitas:cardano.node.ip-producers:Notice:5] [2021-05-23 17:03:16.23 UTC] IPSubscriptionTarget {ispIps = [165.232.138.247:6000], ispValency = 1}

sudo systemctl status cardano-node

What is the uptime?
also type top how is the cpu? 100%?

Hi!

As @Alexd1985 suggesting the node probably in startup phase when trying to open the databases…
check this topic where you can see all the logs of a successfully started node

Hello Alex,

here is the status of the node

May 23 18:02:52 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:02:52.06 UTC] Replayed block: slot SlotNo 26978383 of At (SlotNo 27568343)
May 23 18:06:34 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:06:34.86 UTC] block replay progress (%) = 97.9
May 23 18:09:06 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:09:06.50 UTC] block replay progress (%) = 97.9
May 23 18:10:57 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:10:57.13 UTC] block replay progress (%) = 97.9
May 23 18:11:37 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:11:37.83 UTC] before next, messages elided = 26978419
May 23 18:11:37 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:11:37.85 UTC] Replayed block: slot SlotNo 26999972 of At (SlotNo 27568343)
May 23 18:11:38 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:11:38.83 UTC] Replayed block: slot SlotNo 26999985 of At (SlotNo 27568343)
May 23 18:11:58 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:499] [2021-05-23 18:11:58.27 UTC] before next, messages elided = 27000002
May 23 18:11:58 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:499] [2021-05-23 18:11:58.28 UTC] Replayed block: slot SlotNo 27000559 of At (SlotNo 27568343)
May 23 18:11:59 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:11:58.48 UTC] Replayed block: slot SlotNo 27000572 of At (SlotNo 27568343)

The uptime is about 20mins now. However, before that it was 17 hours. and it was in the same state

top - 18:18:06 up  1:38,  1 user,  load average: 2.72, 2.80, 2.30
Tasks: 104 total,   2 running, 102 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.5 us, 32.7 sy,  0.0 ni, 22.2 id, 36.9 wa,  0.0 hi,  0.2 si,  7.6 st
MiB Mem :   3936.2 total,     87.1 free,   3812.6 used,     36.5 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.     18.4 avail Mem

What did u changed after the nodes were synced?

absolutely nothing. When I woke up they were like that.

image

Current log on the node.

-- Logs begin at Fri 2021-05-21 21:49:52 UTC. --
May 23 18:15:10 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:15:10.79 UTC] block replay progress (%) = 98.0
May 23 18:16:58 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:499] [2021-05-23 18:16:58.41 UTC] before next, messages elided = 27000595
May 23 18:16:58 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:499] [2021-05-23 18:16:58.41 UTC] Replayed block: slot SlotNo 27008300 of At (SlotNo 27568343)
May 23 18:16:58 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:16:58.57 UTC] Replayed block: slot SlotNo 27008321 of At (SlotNo 27568343)
May 23 18:40:45 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:40:45.83 UTC] before next, messages elided = 27008386
May 23 18:40:46 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:40:45.88 UTC] Replayed block: slot SlotNo 27021589 of At (SlotNo 27568343)
May 23 18:40:46 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:40:46.37 UTC] Replayed block: slot SlotNo 27021593 of At (SlotNo 27568343)
May 23 18:47:03 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:499] [2021-05-23 18:47:01.11 UTC] before next, messages elided = 27021605
May 23 18:47:03 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:499] [2021-05-23 18:47:01.41 UTC] Replayed block: slot SlotNo 27023597 of At (SlotNo 27568343)
May 23 18:47:18 Tortilla cardano-node[31612]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 18:47:14.73 UTC] Replayed block: slot SlotNo 27023641 of At (SlotNo 27568343)

wait till you have this log statement:
“Opened db with immutable tip”

this one? what does it mean?

May 23 20:47:06 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:06.86 UTC] Opened imm db with immutable tip at bf4362dcf1c4cb715d3cadc9fdf1809c06ae12a0600be4512d84deaf5e4b6cb0 at slot 27568343 and chunk 1276
May 23 20:47:08 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:08.79 UTC] Opened vol db
May 23 20:47:27 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:27.10 UTC] Replaying ledger from snapshot DiskSnapshot {dsNumber = 26804209, dsSuffix = Nothing} at bec3602292b1af1d0bb481a6c706db3f2690d463ca3a8427c1afca6b4fc7fb68 at slot 26804209

this came after

May 23 20:47:27 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:27.32 UTC] Replayed block: slot SlotNo 26804236 of At (SlotNo 27568343)
May 23 20:47:27 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:27.50 UTC] before next, messages elided = 26804241
May 23 20:47:27 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:27.50 UTC] Replayed block: slot SlotNo 26805594 of At (SlotNo 27568343)
May 23 20:47:27 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:27.51 UTC] Replayed block: slot SlotNo 26805597 of At (SlotNo 27568343)
May 23 20:47:28 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:28.40 UTC] block replay progress (%) = 97.3
May 23 20:47:28 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:28.90 UTC] block replay progress (%) = 97.3
May 23 20:47:29 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:29.53 UTC] before next, messages elided = 26805629
May 23 20:47:29 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:29.53 UTC] Replayed block: slot SlotNo 26827101 of At (SlotNo 27568343)
May 23 20:47:29 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:29.53 UTC] Replayed block: slot SlotNo 26827164 of At (SlotNo 27568343)
May 23 20:47:30 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:30.00 UTC] block replay progress (%) = 97.3
May 23 20:47:31 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:31.07 UTC] block replay progress (%) = 97.4
May 23 20:47:31 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:31.26 UTC] block replay progress (%) = 97.4
May 23 20:47:31 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:31.54 UTC] before next, messages elided = 26827232
May 23 20:47:31 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:31.54 UTC] Replayed block: slot SlotNo 26848742 of At (SlotNo 27568343)
May 23 20:47:31 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:31.54 UTC] Replayed block: slot SlotNo 26848771 of At (SlotNo 27568343)
May 23 20:47:33 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:33.37 UTC] before next, messages elided = 26848804
May 23 20:47:33 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:33.37 UTC] Replayed block: slot SlotNo 26870366 of At (SlotNo 27568343)
May 23 20:47:33 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:33.38 UTC] Replayed block: slot SlotNo 26870387 of At (SlotNo 27568343)
May 23 20:47:33 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:33.45 UTC] block replay progress (%) = 97.5
May 23 20:47:33 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:33.67 UTC] block replay progress (%) = 97.5
May 23 20:47:33 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:33.96 UTC] block replay progress (%) = 97.5
May 23 20:47:34 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:34.10 UTC] block replay progress (%) = 97.5
May 23 20:47:34 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:34.90 UTC] before next, messages elided = 26870454
May 23 20:47:34 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:34.90 UTC] Replayed block: slot SlotNo 26891943 of At (SlotNo 27568343)
May 23 20:47:34 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:34.90 UTC] Replayed block: slot SlotNo 26891972 of At (SlotNo 27568343)
May 23 20:47:36 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:36.16 UTC] block replay progress (%) = 97.6
May 23 20:47:36 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:36.18 UTC] before next, messages elided = 26892045
May 23 20:47:36 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:36.18 UTC] Replayed block: slot SlotNo 26913554 of At (SlotNo 27568343)
May 23 20:47:36 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:36.20 UTC] Replayed block: slot SlotNo 26913577 of At (SlotNo 27568343)
May 23 20:47:37 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:37.05 UTC] block replay progress (%) = 97.7
May 23 20:47:37 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:37.62 UTC] before next, messages elided = 26913628
May 23 20:47:37 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:37.62 UTC] Replayed block: slot SlotNo 26935108 of At (SlotNo 27568343)
May 23 20:47:37 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:37.63 UTC] Replayed block: slot SlotNo 26935129 of At (SlotNo 27568343)
May 23 20:47:39 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:39.96 UTC] before next, messages elided = 26935216
May 23 20:47:39 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:39.96 UTC] Replayed block: slot SlotNo 26956756 of At (SlotNo 27568343)
May 23 20:47:39 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:39.97 UTC] Replayed block: slot SlotNo 26956786 of At (SlotNo 27568343)
May 23 20:47:53 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:53.63 UTC] Replayed block: slot SlotNo 26958246 of At (SlotNo 27568343)
May 23 20:47:53 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:53.64 UTC] Replayed block: slot SlotNo 26958424 of At (SlotNo 27568343)
May 23 20:47:53 Tortilla cardano-node[35385]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-23 20:47:53.79 UTC] block replay progress (%) = 97.8
May 23 20:48:00 Tortilla cardano-node[35369]: /home/salsa/cardano-my-node/startBlockProducingNode.sh: line 9: 35385 Killed                  /usr/local/bin/cardano-node run --topology ${TOPOLOGY} --database-path ${DB_PATH} --socket-path ${SOCKET_PATH} --host-addr ${HOSTADDR} --port ${PORT} --config ${CONFIG}
May 23 20:48:08 Tortilla cardano-node[35426]: Listening on http://127.0.0.1:12798
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.networkMagic:Notice:5] [2021-05-23 20:48:08.46 UTC] NetworkMagic 764824073
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.protocol:Notice:5] [2021-05-23 20:48:08.46 UTC] Byron; Shelley
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.version:Notice:5] [2021-05-23 20:48:08.46 UTC] 1.27.0
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.commit:Notice:5] [2021-05-23 20:48:08.46 UTC] 8fe46140a52810b6ca456be01d652ca08fe730bf
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.nodeStartTime:Notice:5] [2021-05-23 20:48:08.46 UTC] 2021-05-23 20:48:08.462959117 UTC
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.systemStartTime:Notice:5] [2021-05-23 20:48:08.46 UTC] 2017-09-23 21:44:51 UTC
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotLengthByron:Notice:5] [2021-05-23 20:48:08.46 UTC] 20s
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.epochLengthByron:Notice:5] [2021-05-23 20:48:08.46 UTC] 21600
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotLengthShelley:Notice:5] [2021-05-23 20:48:08.46 UTC] 1s
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.epochLengthShelley:Notice:5] [2021-05-23 20:48:08.46 UTC] 432000
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotsPerKESPeriodShelley:Notice:5] [2021-05-23 20:48:08.46 UTC] 129600
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotLengthAllegra:Notice:5] [2021-05-23 20:48:08.46 UTC] 1s
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.epochLengthAllegra:Notice:5] [2021-05-23 20:48:08.46 UTC] 432000
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotsPerKESPeriodAllegra:Notice:5] [2021-05-23 20:48:08.46 UTC] 129600
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotLengthMary:Notice:5] [2021-05-23 20:48:08.46 UTC] 1s
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.epochLengthMary:Notice:5] [2021-05-23 20:48:08.46 UTC] 432000
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.basicInfo.slotsPerKESPeriodMary:Notice:5] [2021-05-23 20:48:08.46 UTC] 129600
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.addresses:Notice:5] [2021-05-23 20:48:08.46 UTC] [SocketInfo 0.0.0.0:6000]
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.diffusion-mode:Notice:5] [2021-05-23 20:48:08.46 UTC] InitiatorAndResponderDiffusionMode
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.dns-producers:Notice:5] [2021-05-23 20:48:08.46 UTC] []
May 23 20:48:09 Tortilla cardano-node[35426]: [Tortilla:cardano.node.ip-producers:Notice:5] [2021-05-23 20:48:08.46 UTC] IPSubscriptionTarget {ispIps = [143.198.137.123:6000], ispValency = 1}

I believe the node keeps restarting because when I look on the guild LiveView, the uptime often restarts to 0

no, - this one:
“Opened db with immutable tip”

So how much time you let the node run in starting phase before killed it?

What is the hw configuration? I think it’s not enough

DigitalOcean
2 CPU
4GB
80GB ssd

I just upgrade to double the specs, and it is showing the same on the logs.

I’ve had the exact same problem, after now successfully perform a number of upgrades since 1.25.x.

My Relay didn’t start (stuck in starting) after 12 hrs overnight, and CPU > 100% in top. Also gLiveView has gone weird around reporting up time etc. Note - simply built upgraded node, stopped, then restarted.

I’m on 4 vCPUs
8GB / 80GB Disk

I’m setting up a new node as we speak, in case we can’t solve by epoch end.

Things seen to have moved a bit after restarting the server a few times and upgrading the resources.

this is what I am getting now. The restarting subscriptions just keep repeating itself now.

- Logs begin at Fri 2021-05-21 14:49:52 PDT. --
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.basicInfo.epochLengthMary:Notice:5] [2021-05-24 00:42:21.59 UTC] 432000
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.basicInfo.slotsPerKESPeriodMary:Notice:5] [2021-05-24 00:42:21.59 UTC] 129600
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.addresses:Notice:5] [2021-05-24 00:42:21.59 UTC] [SocketInfo 0.0.0.0:6000]
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.diffusion-mode:Notice:5] [2021-05-24 00:42:21.59 UTC] InitiatorAndResponderDiffusionMode
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.dns-producers:Notice:5] [2021-05-24 00:42:21.59 UTC] []
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.ip-producers:Notice:5] [2021-05-24 00:42:21.59 UTC] IPSubscriptionTarget {ispIps = [143.198.148.52:6000], ispValency = 1}
May 23 17:42:22 Tortilla cardano-node[743]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-24 00:42:22.49 UTC] Opened imm db with immutable tip at e3657dc8e43850d1b0d6db22f662947ab57efa7ad78ceea84f1068d9f5242427 at slot 27573291 and chunk 1276
May 23 17:42:23 Tortilla cardano-node[743]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-24 00:42:23.32 UTC] Opened vol db
May 23 17:42:40 Tortilla cardano-node[743]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-24 00:42:40.99 UTC] Replaying ledger from snapshot DiskSnapshot {dsNumber = 27573291, dsSuffix = Nothing} at e3657dc8e43850d1b0d6db22f662947ab57efa7ad78ceea84f1068d9f5242427 at slot 27573291
May 23 17:42:40 Tortilla cardano-node[743]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-24 00:42:40.99 UTC] Opened lgr db
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.ChainDB:Info:5] [2021-05-24 00:45:28.33 UTC] Opened db with immutable tip at e3657dc8e43850d1b0d6db22f662947ab57efa7ad78ceea84f1068d9f5242427 at slot 27573291 and tip 591d4db501b22f5df9bf4a378c1767d4838a76354cb4bf890c34140bb0c4de57 at slot 27619564
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:241] [2021-05-24 00:45:28.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Trying to connect to 143.198.148.52:6000
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:245] [2021-05-24 00:45:28.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Connection Attempt Start, destination 143.198.148.52:6000
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Notice:241] [2021-05-24 00:45:28.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Waiting 0.025s before attempting a new connection
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Notice:245] [2021-05-24 00:45:28.34 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Connection Attempt End, destination 143.198.148.52:6000 outcome: ConnectSuccessLast
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:245] [2021-05-24 00:45:28.35 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Closed socket to 143.198.148.52:6000
May 23 17:45:28 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Error:241] [2021-05-24 00:45:28.36 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Failed to start all required subscriptions
May 23 17:45:29 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:241] [2021-05-24 00:45:29.36 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Restarting Subscription after 1.026041001s desired valency 1 current valency 0
May 23 17:45:38 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:241] [2021-05-24 00:45:38.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Trying to connect to 143.198.148.52:6000
May 23 17:45:38 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:264] [2021-05-24 00:45:38.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Connection Attempt Start, destination 143.198.148.52:6000
May 23 17:45:38 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Notice:241] [2021-05-24 00:45:38.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Waiting 0.025s before attempting a new connection
May 23 17:45:38 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Notice:264] [2021-05-24 00:45:38.33 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Connection Attempt End, destination 143.198.148.52:6000 outcome: ConnectSuccessLast
May 23 17:45:38 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:264] [2021-05-24 00:45:38.34 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Closed socket to 143.198.148.52:6000
May 23 17:45:38 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Error:241] [2021-05-24 00:45:38.36 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Failed to start all required subscriptions
May 23 17:45:39 Tortilla cardano-node[743]: [Tortilla:cardano.node.IpSubscription:Info:241] [2021-05-24 00:45:39.36 UTC] IPs: 0.0.0.0:0 [143.198.148.52:6000] Restarting Subscription after 1.026800443s desired valency 1 current valency 0

Now you have all the databases opened…
I would extend the relay addresses in the topology.json file since the default address seems not enough…