Thanks for the great discussion!
Colin: Supports running pools as long-term viable businesses
Pal: Are we talking about operation or ownership? Because those two can be different and it would mean different things in both scenarios (when operator=owner, probably small businesses versus operator!=owner probably whales outsource their operation to ops, or in-house operation.).
Shawn: It’s a good idea to take all 3 perspectives into consideration (owner, operator, owner/operator).
Colin: Does not forcibly kill off pools outside K
Pal: I think it just makes a rational assumption that a pool /w that characteristic would not survive long term and acting based on that assumption.
Shawn: The only reason a pool ranked at k + 1 would not survive long term is because the current rankings system is designed to kill them off. There is likely no real difference between pool k and pool k + 1. This goes back to what I believe is a false assumption that an “optimal” network has exactly k fully saturated pools and all other pools with no delegation.
Pal: Also, a pool /w 0 delegated stake (but have some pledge) could earn reward weighted by a0 anyway (around ~5% per annum) assuming they create their blocks when they were expected i.e. scheduled, but removing them from the favourable pools.
Shawn: There is very little reason to go through all the trouble and cost to run a pool with no delegation unless you are a whale with massive pledge. It’s more efficient to delegate to a pool with pledge higher than your delegation that has low fees.
Pal: In simple words, it could be eliminated if the pools ROI would not follow the characteristics that can be seen in the attached graph below, but would be weighted by pledge. I.e. it would not incentives to create multiple pools /w smaller pledge because if they would create n number of pools /w low pledge those all pools would have smaller yield then they would have created only pool. Ofc, their ROI should be higher than the delegators ROI for incentivising them for operation. Anyway, that killing mainly applies to the small businesses (op=owner) who do not have enough pledge for compensating their cost by rewards i.e. cost >=pool reward. But, it simple means a mum can run a pool at home running the cardano-node on an RPi4 /w relative small pledge and could gain yearly ~5% ROI on her pledge.
Shawn: I believe those small businesses being killed off are much more valuable to running a stable and growing network than a few hobbyists running nodes at home.
Pal: So, killing of pools simply means they will earn only after their pledge (~5% per annum), and if that reward is not enough to cover their cost plus have some yield, then they can have an option to delegate their stake (pledge at that time) to other pools and shut down their operation if they want.
Shawn: This is precisely the problem. There is no reason to kill off this valuable resource that will especially be needed as Goguen is released and there are much higher demands on the network. This seems very short-sighted and destructive for what appears to be solely an academic mathematical goal.
Pal: It’s like “private pool” in a sense that they do not have any delegated stake but their pledge. Large businesses, should have enough pledge to cover their operational cost (usually the 340ADA for an epoch), if they do cannot then they are not large businesses. So, It think, it does not mean killing of pools outside of k, but simple means the rest would became as public private-pools, but killed as a business.
Shawn: Which does essentially mean they are killed off. I don’t think we want to run the world financial operating system on a few preselected early winners and a few hobbyists.
Colin: Does not incentivize multiple new pools creation to game the system by having some pools “get lucky”
Pal: Then the current RSS should be amended, as have a look at this graph, which represents the pools ROI propotional to their pledge. As you can see the sweet spot (not in economical sense) is the 0.000531 (minimum of the function), which in ADA is 0.000531 Tc=16,885,800 meaning that is the minimum of the rewards. So, the whales are incentivised to full pledge to gain max reward while the small pool incentivised split their pledge (like 1PCT ZZZ BLOOM etc. pool). A small pool /w 1.3m will be able to gain maximum reward as a fully pledged pool. You can see that this function is not optimal for controlling small pools, but incentivise them to split their pledge. You can check functions and tune it parameters of the plot in desmos what I have just created for explaining/describing the issue. Desmos ROI (Pledge Versus Reward)
Shawn: I believe this may be addressed by my Curve Pledge Benefit CIP.
Colin: The implication is we may need to evaluate the fitness of more than one measure:
the realized rewards from last-n epochs (including the effects of luck)
the expected returns for next epoch (excluding luck)
the expected returns for next-n epoch (excluding luck, but assuming saturation)
a long term measure than accounts for maximizing rewards and system stability
Pal: Yes, you are right, as luck is contradict for small pools if we’re considering performance with an assumption as if they would be fully saturated. Because in that case their performance would be nearly 100% if they’re playing honest (which should not be relevant for ranking calculation). While at the moment, they’re suffering from luck based on their low pledge and/or stake. We should assume, that performance nearly will be constant for pools have some saturation (it now depends on the d too)
Colin: I also suggest some measure as to whether variability in block creation is due to random chance vs a signal that a pool is not doing a good job creating its blocks. For example, 95% certain that underproduction is due to poor pool operation rather than random chance. (Assertion: block over-production should be disregarded in all circumstances, as noise.)
Pal: I think that’s a false assumption with the Haskell code base and the with current settings, as we need to distinguish between the small pools (in a sense of stakes) which are suffering from d and the pools having some high amount of stake as the threshold for creating a block is depends on d and delegated stakes (pledge included).
Shawn: The randomness has a bigger impact on smaller pools so you need a longer timeframe to reduce the variability due to randomness. Excluding overproduction probably does not help much as you are chopping off the half of the bell curve above the expected number of blocks but you still have the variability on the lower side of the bell curve which probably takes (almost) as long to determine apparent performance.
Pal: Also, If a pool has a 50% chance of generating a block in an epoch it will create (on average) a block in every two epochs, and it does not mean that it;s playing nasty, even that is it does not have luck. So, a pool must have a certain stake to be successful part of staking i.e. 1/(21600*d) currently around 4.5m ADA (450K USD stake pledge included) of around 1.5m when d=0. It’s a bit more complex, but if a pool want be safe then it should pledge that amount which is not really feasible for small businesses. Of course IOG tried to help the pool’s bootstrap but they are and won’t be favored if initially the luck is considered in the non-myopic ranking for the pools.
Shawn: If you have a non-myopic view of apparent performance that is the average of the actual blocks divided by the expected blocks over an initial period (say 10 epochs) and then a function as I described that values the current apparent performance appropriately to the ongoing historical apparent performance you can get an idea of the pool’s performance that includes it’s entire history but is weighted towards the most recent epochs.