Block producing node is dropping in connection to relay

Staking & Delegation Operate a Stake Pool
1

I have a strange problem. After the update of the nodes to 1.26.1. “Already updated to 1.26.2”
The block-producing node is dropping IN connection to relay in a repeating pattern. After I restart the Linux os of the BP node everything again working OK, Аnd after approximately 24 hours this is repeated.

Has anyone had a similar problem, and know how to fix it, because there is nothing strange in the logs that leads to this behavior?

I hosting the nodes in DigitalOcean droplets:

2x (
Ubuntu 20.04 (LTS) x64
4 vCPUs
8GB RAM
80GB SSD Disk
)

2

But, the Relay… isn’t supposed to restart itself once/24 hours?

1 Like
3

Maybe because you are running the service to restart the nodes every 24 hours? Cntools has a script for that for systemd

1 Like
4

I am currently monitoring the pool and restarting only the BP node manually only when this problem occurs. Is it necessary to restart nodes in some time period ??

5

do they need to be restarted at a certain time?

6

nope, it was a question… if u are using cntools + topology updater + systemd then the nodes are automatically restarted by default once/24 hours

if u are not using this combination then it’s not the case…

Anyway… when the BP dropping the connectivity what is the status of the Relay?

7

It’s running normally. Receives tx’s. And all looks good. what I noticed is that the RAM memory of the relay was 97%

8

yeah, and the Relay not restarted by itself due to memory issue?

Try to set in the relay config file TraceMempool=false and restart the Relay…
you will not see the tx processed on Relay (only on BP) but at least u will not have memory issues anymore…
or try to restore the Relay once/12 hours (this way the memory will be cleaned before to reach 97%)

1 Like
9

I have enabled swap… And may be this is the reason not restarting. Is this the problem cause BP node to dropping connection to relay ?

10

yes, can be

1 Like
11

I will try. Thanks.

12

I (and some others) have noticed the same effect with the random connection dropouts and stutters.

Try Alex’s great advice:

“Try to set in the relay config file TraceMempool=false and restart the Relay…
you will not see the tx processed on Relay (only on BP) but at least u will not have memory issues anymore…”.

I did that for my BP’s last epoch and it is making a HUGE difference (was already doing on the relays). Like Alex says, you won’t see the TX’s increase on your relays or BP so you should only disable that setting once you are confident that your setup is working correctly. It worked for me even if there may not be any direct correlation between that setting and the connection problem.

2 Likes
13

I tried this, but don’t resolve the issue. It has improved in RAM usage, but after near 20 hours, the connection dropped again. I think the problem is in the BP node operating system. Because it resolves the problem only when I restart the operating system of the BP node, if I restart only Cardano-Node process it’s unable to create IN connection.