Relay node syncing issue - 1.27.0

Hello,

I am trying to setup a new pool. I was syncing my relay node and it was around 92% the last time I checked it. I came back a while after to check if it was finished and I got this screen…

image

Not syncing anymore, the uptime restarted, and it is showing starting… message since then. Before it was syncing for more than 15 hours and seemed all correct. Do I have to assume it has been synced or that there was a mistake and I have to restart again?

Is there any message or status that indicates when the syncing is correct or is it likely that there was a mistake/restart so not full syncing done?

Can you please help me on this issue?

Thank you, I am new, sorry if it is a too naive issue.

Best regards,

Alex

Hi!

check the node logs whether all the databases opened or not?

Also share what is the memory size of your system…

Hello,

Thank you for your answer. This is what I get when I run

image

According with what it shows, seems similar to the information you shared with me. There is no producer not yet, so that is maybe why there is no more information after the InitiatorAndResponderDiffusionMode message.

The memory size is the 4GB droplet from DigitalOcean.

very similar but I can not see the database opening related logs…

[X:cardano.node.ChainDB:Info:5] [2021-05-12 16:28:05.48 UTC] Opened imm db
...
[X:cardano.node.ChainDB:Info:5] [2021-05-12 16:28:06.99 UTC] Opened vol db
...
[X:cardano.node.ChainDB:Info:5] [2021-05-12 16:28:45.70 UTC] Opened lgr db

But u can see the killing message

yes, I already linked another topic but we could link more with similar content…
So unfortunately with 1.27.0 the 4GB RAM is not enough… - what do you think @Alexd1985

like this one:

Seems some part restarted and it is generating a new log, I will share it now once it is finished…
But If 4GB is not enough, this will make it intractable in the long run…small pool operators might not be able to afford 16+GB servers (especially if 3 are required).

8G min + 2-4G swap required

4G will not be enough

Here is the new log…still not finished and not showing the kill message…

And then, after some time, this is the complete log…showing the kill message…

Could the kill message only an issue memory? It could not be caused by a bug or issue in the syncing process?

just wait - either the system will kill it, or the ledger database will be opened…
if the syatem kills it - it is likely due to high memory usage…
what you can do is to roll back to 1.26.2 or get 8 GB of RAM

I waited, the output is above. This is what is killed…

It seems it comes from line 9 of the Bash script and showing some number. Could it just be a memory related issue?

if you missed the update on my recent post

Thank you, your information is very helpful for me. But what is the issue if I roll back to 1.26.2? I will have to update it at some point in the future, isn’t it?

Sorry but I am new, so I do not know what actually 1.27.0 improves/changes from 1.26.2…the ledger is the same but there are some other updates? So they can be used interchangeably?

sometimes there are changes which makes a rollback harder or impossible… but there is not such change between 1.26.2 and 1.27.0
so right now the recommended system memory is 8 GB

so might you need to upgrade anyway if there will be no improvements on memory usage…

Just a personal observation for those who might not know. occasion missed slots in Graphana is the sign of performance issue. (At least one of them)

1 Like

Sorry @Anton_R I dont understand, what do you mean? Is this another indicator I should switch to 8GB memory machine?

If you are running 1.27.0 it is strongly recommended.

Thanks @laplasz. As I see, from 1.26.X, the requirements went up from 4GB to 8GB of memory. I thought it was recommended, but It might be a critical issue then. How can I roll back to 1.26.2?

Thanks @ToTheMoonADA but if it crashes, then it becomes compulsory. What if I use 1.26.2? Might work well with 4 GB? How can I roll back to it?