1.26.2 high memory usage

Hi, I noticed after upgrading to 1.26.2 that the memory usage on relays is comparable to the 1.25.1, meaning that is very high if tracemempool is kept active. Did anyone upgrading from 1.26.1 to 1.26.2 noticed an increase in memory usage?
Also on BP the RAM usage is close to 4Gb, higher than before.

1 Like

I have traceMempool off on all my nodes and I also see an increase in memory usage as the hours go by. After 12 hours or so, I can see a noticible difference / increase in memory used. Almost like there is a slow memory leak happening somewhere. I am now in the habit of restarting nodes about once a day just to be safe.

1 Like

I see an increase in memory usage over time on BP. After reboot: 2.5GB, reaches almost 4GB after a couple of hours. Also MissedSlots start to increase when memory peaks.
I’ll disable tracemempool also on BP to see if this helps. Not happy!!!

1 Like

After some time it’s clear that the memory usage increases over time for both relays and bp, even with tracemempool disabled. Anyone else is noticing this behavior?

1 Like

I have noticed much higher memory usage with 1.26.2 as well. Something seems off. I’ve had a couple of commands crash cardano on my relay and producer node, running out of memory.

example command causing crash:
cardano-cli query ledger-state --mainnet | grep publicKey | grep <pool id>

I’ve set TraceMempool to false on the relay this morning to test. I should know more within a day about whether this helps.

Have you resolved the issue with missed slots? I’ve found that bp node starts missing slots after about 9 - 12 hours and reaching 5 GB memory consumption. Looks a bit like a memory leak somewhere.

My relay is sitting at 3GB this morning after a restart, while the block producer is around 5GB. The relay will creep or jump up to 5GB at some point today I’m sure. Something is inefficient here and needs optimization. This essentially doubles the infrastructure costs per instance moving from 4GB to 8GB RAM requirement.

same here, it crashes and I’m on 8gb ram

Today, while generating ledger state on a 1.26.2 node, memory usage upto 14.5GB.

I have retired my stake pool due to the exponentially increasing requirements of Cardano. When Jormungandr was used, it could all be run from one machine with much less memory.

Now, even 8GB isn’t enough. CPU was at 100% on the relay due to lack of memory, lots of errors about invalid Txs not in the UTXO set (mempool tracing etc on), swap usage was at 100%. I really haven’t got the time nor the energy to run this any more.

The node running 1.26.2 was fine (block producer), the relay running 1.26.1 (which 1.26.2 was deemed not needed for relays) was really not having a good time at all.