Restart producer node - 18 minutes to sync

[stiiiif]

Hi Beautiful Community!

When I kill my block producer node and restart it, it take around 18 minutes to re-sync.
I’m wondering how I could reduce that time, if that’s normal, if it depends on the number of cores, or else?

Thanks for you input

stef

[COSDpool]

welcome @stiiiif … this was typical of earlier versions of cardano-node but with recent versions the sync time has been cut down to mere seconds. First check that you have this version installed, because a lot of install documentation still points to old versions (all releases are here):

Release Cardano Node 1.19.1 · input-output-hk/cardano-node

[lauris]

depends how you kill.

best one from live view is to hit the “q” letter.
if you are using kill or killall - then use SIGINT signal, which ends cardano-node process gracefully. if you are sending killing the process without sigint, then cardano-node process is not able to close db and in some cases remove socket file, so next time you start, cardano-node process has to check db for inconsistencies.

[COSDpool]

thanks @lauris that’s good to know. In version 1.18.0 this was not the case… SIGINT had the same effect as SIGHUP, SIGKILL etc… crash and burn no matter what, with ledger having to catch up from the last checkpoint. We reported this here & sounds like it’s been fixed:

How to gracefully restart cardano-node?

No luck finding an answer for this on IOHK Github, Telegram, or elsewhere on forum (my choices of search keywords were probably not very effective), or trial & error: TL;DR we need a way to send a signal, a packet, some json, add/delete a file, or anything that will cause cardano-node to “exit nicely” before termination, so it doesn’t have to go through a huge DB consistency check & cleanup the next time it starts. cardano-node is working fine for us & syncing all the way to the tip, but I’m r…

Or maybe we’ve seen the huge improvement ourselves in restart speed since our systemd script was set to use SIGINT from the beginning, even though version 1.18.0 didn’t catch that signal at the time (@stiiiif you could also try implementing this to see if it speeds up your restarts):

Pool maintenance routines

Here are some things I needed to do to feel comfortable taking my eyes off the stake pool node log files every once in a while, after the initial days of staring at them like that creepy guy in The Matrix: Make browser bookmarks to general performance metrics for cloud resources for each node (like DigitalOcean droplet graphs, a bit simple minded but effective; or the AWS CloudWatch pages, not the detailed paid upgrade but just the free ones) to be sure RAM or CPU use isn’t clipping that the m…

[stiiiif]

Thanks to both of you!