responding to a query from @Meritus by Twitter PM, so it can be held up for review & benefit others in the same situation if it’s considered valid:
+RTS -N -I0.1 -Iw39600 -A32M -AL256M -n16m -F0.3 -O12G -M23G -c99 -Mgrace=1G -C0 -RTS
Upgrading to 24GB RAM, what might you change about that to make more use of the effort?
The -O option linearly relates to the amount of real memory on the node… with a slope of 1, so every increase in memory should increase the -O option by the same amount. I reverse engineered from the values @_2072 was using above that -O should be set to the difference between real memory and the size of “program” data used by the Cardano Node (roughly the amount of memory you see it using when it first starts up). This difference is the size to which the “heap” of GHC data can grow before a garbage collection is forced.
The value of 12GB above was calculated from (roughly) 4GB used by the Cardano Node itself taken away from our 16GB of real memory. However it feels more comfortable to reduce this threshold a bit to make sure there’s always at least ½GB of free memory, even across an epoch boundary when relays will use more memory… so this can be reduced by exactly the amount of memory you’d like to see kept free in the free command during peak periods.
So if I were to add 8GB of memory to a 16GB system, I would want to make 8GB more memory available to the “heap” and change the -O option to -O20G… or maybe 19.5G or lower, once I saw how close to the memory limit it was actually getting.
The -M option is an upper limit on heap size before the node tries to shut down. I assumed this option was added above in case the node runs away with heap memory allocation: to ensure it doesn’t crash leaving a corrupted data structure & long restart time. Again reverse engineering from the original author’s posts, I could see every change in -O was accompanied by a 2x increase in -M… which also makes sense to me because it maintains a safety margin for heap growth roughly equal to the usual heap size itself.
Setting -M too low could cause the node to crash prematurely, while it should be set no higher than the amount of real memory on the system plus the amount of swap (because obviously the “heap” needs to fit within the sum of real + virtual memory). So if you agree with this rationale so far, adding 8GB to the system would add 16GB to the -M figure (or maybe less) for a new value of -M39G on a 24GB system (with at least 16GB swap).
Those are the only two options I would change if the amount of memory on a system changed. Other SPOs trying these configurations for different memory sizes are welcome to post their experiences as follow up 