BP suddenly not processing TXs but Relay yes

Hi Everyone, last Friday my BP stopped processing TXs and lost sync with the blockchain. My Relay node is working fine and if I reboot both, the BP syncs but then it stops processing TXs again.
I don’t see any errors and all configuration seems to be fine (as this was working).
I’m running topologyUpdater and it says glad to stay with us
I used cntools to configure everything.
Ticker: R2D2
Setup is 2 servers 2CPU x 16GB on public cloud.
Any idea what can it be?


Thanks.

I see syncing message for the producer
It looks like the Producer is losing the connection with the relay

Yes, every time I reboot them they sync and then BP looses the connection

Try to ping the relay… do u have any packet lost?

Im not losing any packages, I’m connecting using the private network.
Screen Shot 2021-07-19 at 13.57.25

Try journalctl -e -f -u cnode

Perhaps it will show something

Everything seems normal, both relay and BP shows this:

“WARN: A prior running Cardano node was not cleanly shutdown, socket file still exists. Cleaning up.
Jul 19 16:23:32 virtualserver02 cnode[705472]: Listening on http://0.0.0.0:12798

I’m wondering if maybe is because of topologyUpdater.sh on Relay, I saw a post where someone had a problem with format IP,PORT (with comma) and I changed it to : (two points) that was an hour ago, I will see if that changes anything in the next 2 hours.

Also after some time the BP just dropped OUT connections and now I have only relay on IN

Yeah… something is not ok… why the Producer drop the connection… try to add also the IOHK nodes inside Producer topology file… and check if also will drop the connections with IOhK nodes

{
“addr”: “relays-new.cardano-mainnet.iohk.io”,
“port”: 3001,
“valency”: 2
}

It’s just for out, u will not need to change any fw rules

This is strange, I had to enable the public network for the BP to reach IOHK nodes, and once I did it, it started to process transactions… Is now connected to IOHK nodes and my Relay, and processing TXs on IN via my Relay as it should.
So I think there is a problem with my public and private network somehow.

Hmmm… keep it under monitoring

Then delete the IOHK nodes and try to connect the BP to relay via public IP

If is ok then switch back to private and see if the issue will re-appear

Thanks Alex, I will do, I will keep the public network for some hours and then will try to disable it again. Anyways I also have a firewall, but everything is an extra layer of protection.
Thanks again!!!

1 Like