String of orphans

The only time I’ve ever seen more than statistically plausible numbers of orphans in a short time period would be after the number of Hot Peers on our relay co-located with our BP gets up past 300 towards around 350, with over 400 full-duplex connections (at these times the load average of our 12-core machine is still only around 1, so the system itself doesn’t look overloaded).

Checking time sync and propagation delays was also the first thing I thought of: both optimal. So I’m operating under the assumption now that this number of P2P connections isn’t really vetted by IOG, since we only started to see it after our stake increased and I guess we became more attractive as a “ledger peer” proportionally to the square of our ratio to total stake… perhaps a difficult thing to simulate under laboratory conditions.

Since then I’ve set the policy of rebooting that relay (building those Hot Peers up from a small number again) every time I see more than one orphan in a short time period. Generally we can go 100 blocks without losing a slot battle and that seems to restore this performance. So I would be curious whether others have been seen a “string of orphans” with an equivalent number of hot peers / duplex connections, and what they might have done in response.

1 Like