So I was in the process of syncing my node and all of a sudden at 95% I can see now that no more OUT peers are there to sync with. I would also like to mention my node has been syncing for a VERY long time. its been over 24 hours since I started the sync process. Im running my node on Ubuntu 20.04 t2.medium instance on AWS. On my other node (relay node) a different issue occurred where during the sync the socket file seemed to disappear and now when i try to query the tip or run the monitor it returns an error saying the socket file was not found. I don’t know if these things are related but any advice on where to begin the triage here would be super helpful.
I’d also like to mention that I had recently setup 2FA on my nodes although it didn’t seem t affect the node sync when I checked after setting it up initially. I came back 2 hours later and these things had occurred. Not sure if they are related but i wouldn’t think so.
First of all keep the iohk nodes in topology file till the nodes will be 100% synced and 2nd the 2FA installation should not affect the OUT connection
Got it. that explains why my BPN was not receiving any more peers because my relay node failed and that was the only node it was configured to talk with. Now to my second issue, The relay node, which is configured to talk to both my BPN and the IOHK nodes, will not start back up for some reason. It seems to just hang on this point during startup.
May 06 13:12:00 ip-172-31-77-188 cardano-node[525048]: [ip-172-3:cardano.node.diffusion-mode:Notice:5] [2021-05-06 13:12:00.58 UTC] InitiatorAndResponderDiffusionMode
I also can see that there is no socket file in the db location. Any idea on what this might be?
Type
sudo systemctl status cardano-node
journalctl -e -f -u cardano-node
This is the entire log output. Its been this way for the entire time.
May 06 14:45:09 ip-172-31-77-188 systemd[1]: Started Cardano node service.http://127.0.0.1:12798 relays-new.cardano-mainnet.iohk.io ”, dstPort = 3001, dstValency = 2}]
And how it’s looking the glive output?
Guild LiveView version check…
ERROR: CNTools failed to load common env file
Ok, show me the topology file
Try sudo systemctl status cardano-node
{
“Producers”: [
{
“addr”: “My BPN Nodes Ip”,
“port”: 6000,
“valency”: 1
},
{
“addr”: “[relays-new.cardano-mainnet.iohk.io](http://relays-new.cardano-mainnet.iohk.io/)”,
“port”: 3001,
“valency”: 2
}
]
}
I left out my BPN IP address on purpose, theres an IP there in the actual file
sudo systemctl status cardano-node
● cardano-node.service - Cardano node service
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.basicInfo.slotLengthAllegra:Notice:5] [2021-05-06 14:59:18.81 UTC] 1s
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.basicInfo.epochLengthAllegra:Notice:5] [2021-05-06 14:59:18.81 UTC] 432000
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.basicInfo.slotsPerKESPeriodAllegra:Notice:5] [2021-05-06 14:59:18.81 UTC] 129600
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.basicInfo.slotLengthMary:Notice:5] [2021-05-06 14:59:18.81 UTC] 1s
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.basicInfo.epochLengthMary:Notice:5] [2021-05-06 14:59:18.81 UTC] 432000
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.basicInfo.slotsPerKESPeriodMary:Notice:5] [2021-05-06 14:59:18.81 UTC] 129600
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.addresses:Notice:5] [2021-05-06 14:59:18.81 UTC] [SocketInfo 0.0.0.0:6000]
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.diffusion-mode:Notice:5] [2021-05-06 14:59:18.81 UTC] InitiatorAndResponderDiffusionMode
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.dns-producers:Notice:5] [2021-05-06 14:59:18.81 UTC] [DnsSubscriptionTarget {dstDomain = "relays-new.cardano-mai>
May 06 14:59:19 ip-172-31-77-188 cardano-node[1333]: [ip-172-3:cardano.node.ip-producers:Notice:5] [2021-05-06 14:59:18.81 UTC] IPSubscriptionTarget {ispIps = [3.239.0.203:6000], ispValenc>
lines 1-20/20 (END)
namefacedude:
1min 30s ago
I see the relay is restarting
1min 30s ago
try to monitor the node
type more times (3-5 min interval)
What is the hw configuration of the node?
1 Like
Having the same exact issue.
Relay completely syncs then pauses and restarts by itself every five minutes (estimated).
The status says almost exactly what the above says and gets stuck not syncing anymore.
I deleted the DB numerous of times.
I’m running v1.27.0.
Script host --host-addr 0.0.0.0
Config.json set from 127.0.0.1 to 0.0.0.0
Stuck…
Hey Active, the solution for me was to increase the memory size of my instance. I was running a t2.medium on AWS and I upped it to a t3.large. I was running 1.26.2 though so it could be different. Let me know if that works!
I have two R610s with 48gb RAM and two Xeon six core processors each. I only have two 146GB enterprise SAS drives on both. I reverted the config to 127.0.0.1.
The ram sticks are 4gb each… would that be an issue? I’ll look into the BIOS settings to see if I can enable optimization - I still have no luck.
The 4gb ram sticks might be the issue. You have plenty of processing power otherwise. What are the ram sticks for?
They’re servers with 48gb ram.
Basically it’s 4gb ram × 12 so wouldn’t it make it 48gb ram? All same brand and frequency.
But why are u running 1.27.0 version? It’s not official released. Try 1.26.2 and check if u still hVe this issue
Yes still having the same issue. Crashing after syncing or close to it. Ram reaches 5.3gb and crashes. Processor up to 230%…
I switched version same issue… Seems like Cardano likes to use a single card and not take advantage of the others. Unless I’m wrong. It reached a peak of 5.6gb and crashed.