Hi Everyone,
I received a bunch of Grafana alerts on my phone due to a surge in transactions where my pool performance degraded noticeably. I wanted to query the community to see what other SPOs experienced during this event.
A CardanoBits auction event started around 22:00 UTC. I saw a noticeable degradation in my pool performance during this time and wanted to make some changes to prevent future issues. Here is what I saw happen to [NEON] during this time:
CPU was pegged at 95% for minutes at a time
The block producer missed 132 Slots in a 1 hour time period
Transactions Per Second briefly reached 80 Tx/Second!
Transactions in Mempool remained above 350 for a period of time
First off, it’s awesome to see this type of usage on the blockchain but an aws T3.Large instance did not have the ability to handle it. I moved the my block producer from T3.large to C5.xlarge to try and mitigate this as a future risk. For those of you unfamiliar with AWS, T3 is general purpose and C5 is compute optimized but noticeably more expensive.
So… What did everyone else see during this event? I don’t think I’ve seen a transaction surge like this before. I feel this is a good growing pain but something that must be discussed.
