Small SPOs - have you noticed a change in the number of slots being allocated since Epoch 320

Please can any small pool operators with stake in the vicinity of 250k to 350k ADA answer a poll I have put on Twitter. The poll is based on a perceived drop in slot allocations for small pools since Epoch 320?

Link to poll: twitter.com/GrahamsNoPlus1

If you are a small pool operator that fits into the above criteria, please take the time to place a vote on the above poll - poll is open for 7 days up until the 3rd December 2022 - so would really love to get some feedback on this.

1 Like

Hi Russell,

The following topic seems relevant:

Also, please see the topic:

1 Like

Thanks - yes I have been following a number of topics on this - I am just curious if any others have seen a difference since Epoch 320 - my luck has definitely changed and I check for slot allocations every 5 days but mainly 0 allocated even though chances are 1 in 4 each time - yes could be just bad luck but I have heard others are seeing the same thing

Do you still get blocks with low probability?
Our pool was around 200k-300k before, and have about 20 epoch no block

Now our pool have 650k, almost every epoch have block, I don’t know why

I have also >600kADA staked
A block every epoch sounds like very good luck for you! I regularly have 2,3,4 epochs with consecutively no allocated blocks / empty leaderlogs… :frowning:

My understanding it that is should be based entirely on probability - we have an estimated slot allocation of 0.25 or 1 in 4 each Epoch but since Epoch 320 this appears to have changed. I keep getting told that this is just down to luck and we are having a bad run which is entirely possible of course. However I have heard from others that small pools appear to have been impacted since about Epoch 320. This may be totally wrong but I am interested in getting the experience of other small pool owners.

As you can see from our block history things definately appeared to have slowed up since Epoch 320

Yes that can make sense - its not uncommon to get very long runs of black say on a roulette table even thought the chances are roughly 1 in 2. The problem with slot allocation I just have to trust that it is being fair - maybe others with better programming knowledge have looked at the algorithms and are satisfied everything is fair but for me I can only go on what I am experiencing. As you can see I went 31 Epochs in a row with 0 slots (I check every Epoch if I have been allocated one to be sure) - chances of 1 in 4 (based on ~300k in stake) not being selected 31 times in a row is around 0.000134 or 134 in 10,000 chances (if my maths is correct) or 1.34 in 100. Yes that can happen? - so happy for others thoughts on this especially having following periods of 14 Epochs with 0 slots and again another 8 so far with no slots - this hardly helps small pools when such wild variances don’t even themselves out.

After checking for any slots for the next Epoch (380) and finding out yet again that no slot has been allocated I am starting to lose faith in this eco system. I have now had only 3 slot allocations in 59 Epochs - pool configured correctly, kes keys and everything else are fine. 3 slots that were allocated minted without problem so this is definitely an issue around slot allocation.

To be honest I am sick and tired of being told this is just down to luck and I plan to do an analysis on as many pools as possible with stake around 260k to 300k since Epoch 320. I will be looking at their actual blocks minted against expected numbers and I will post the result here. I am past the stage of apologies and being told this is just luck. My delegators and the entire Cardano Community are entitled to know if small pools are being put at a disadvantage.

As an update it is now 3 slots in 60 Epochs with an expected of 0.25. Now just keep telling me its down to luck or someone please address this for the sake of small pools

Who and how?

Have you opened tickets with IOG’s Github or Zendesk (since they will stumble over forum posts only accidentally)?

Have you changed your VRF key, since this mess began?

On the one hand, if there is a problem with the VRF calculation, it has to be very esoteric, since there weren’t loads of reports of SPOs seeing what you see. So (without having looked at the details of the calculation), it might be that only some rare cases of VRF keys produce a bias and with changing it you could already escape it.

On the other hand, it would allow to publish your old VRF private key (after the new one has become active), so that the leadership calculations for all those epochs could be reproduced to really analyse what is happening here.

Yes I have and they said the pool is working fine it is just your luck

No - why would I change it? - everything worked fine in the past - are you saying that this might improve things?

There are a number of SPOs that have reported this

Also this is not about minting blocks - the 3 that were allocated were minted just fine - this is about the allocation which I am claiming is not proportional to the stake right now

What makes you think that I think it were? It is totally clear since your first threads months ago that it is about allocation, not about missing allocated blocks.

Do you have links? In 1.5M Stake - 4 Epochs and no slots, there were some, but they all seem to be back to normal by now.

I’m not saying that I believe there is such a bias, but if (big if) it depends on the VRF key, changing it would help.

But the main reason is that in order to check your claim that something is wrong, it would be good to be able to reproduce your leaderlog calculations. And that needs the private key. That you have to keep private as long as you are using it.

@Terminada and I already proposed back in September to have a detailed look at how the calculation is done in Does an ideal of 0.25 really mean a 25% chance of getting a slot each Epoch - #31 by Terminada and the following posts.

Who should help you how otherwise? With just a vague claim that something seems odd (yes, it does, could still be extraordinarily bad luck, though), it will be very hard to get somebody™ to do anything.

It has to be either shown mathematically, where the problem in the slot lottery is, or shown very clearly with example keys (yours or others) that it is biased (against small pools or against specific keys or whatever).

I appreciate what you are trying to say here and maybe it may have to come down to me releasing the private key - At them moment I am going to look at the results for similar sized pools and see what is happening there

1 Like

I understand that you are trying to help but to be honest this does not really make sense. There appears to be some level of agreement that this is either some kind of bias produced by the vrf keys or this is down to luck - it also appears that both are unlikely - you even say this would be extraordinary bad luck or very rare for it to be vrf bias. I would have thought under the circumstances that this would be of interest to the developers and it is them that should be following up on this. I will raise another ticket with IOG because if there is a potential issue with vrf bias I don’t see why I should be conducting a complicated analysis around vrf analysis which would take considerable time and effort. I am just a SPO who has been doing this for 2 years and contributing to the eco system - I have raised the issue which I will do again with IOG but they need to do more than just put it down to luck. If there is an isolated issue with vrf bias then they surely would want to look into this. Cardano wants to be perfect on every front with peer reviewed papers etc - well lets have IOG take some time and look into this - like I say I will raise this with them again and be guided by vrf key changes if they say so but I would expect some support from them over this. After all I have supported the eco system at considerable cost to me over the years I think it is the least they can do especially when there is a potential bias no matter how remote that chance may be but it is not zero.

Thanks for your help - Ok I understand that we may not see eye to eye on this and that many senior SPOs may not agree with me but I need to have this looked into. Again I appreciate your help but I can’t help getting more and more frustrated at what is turning out to be an unfair system for me.

61 Epochs now and still only 3 slots - I have reported to IOG again and asked for their help but so far they have not responded.

As stated above this must be down to either a really really really unlikely possibility, some form of vrf bias or some other related technical issue that is not related to my pool as this has been checked by IOG before and confirmed to be configured correctly.

It is kind of sad as I will have to tell my delegators to remove their stake and go to another pool as continuing in this way is not fair on them. I guess I will never know after this as there will be no likelihood of every minting a block. Right now I am pretty P%^$ off at how this whole journey has turned out.

1 Like

Here is what I say on my own website for people thinking about delegating to my pool:

"I do not recommend you stake with Terminada pool yet because the protocol does not allow setting the fixed fee any lower than 340 Ada.

With a fixed fee of 340 Ada you will lose too much of your rewards to fees until the pool size is over 10 million Ada."

Try to see the whole thing as a learning experience. I believe that there will be many more opportunities to be gained from running a stake pool in the future. For example, it will give you an opportunity to participate in running side chains and also a front row seat to provide market making services for Babel fees and smart contract apps like AXO. You have some skills and a foothold that others don’t have.

1 Like

Thanks for your comments and I appreciate what you are saying. With the 340 ADA I have always returned the ADA to delegators so as to make us as competitive as possible but this is irrelevant right now as we just don’t get any slot allocations

Yes true it has been a learning process - however there still remains this dilemma as to why my pool is not getting any slot allocations. So far I have had no response from my ticket with IOG and I really think they need to examine this because the likelihood of there being some bias against small pools (or at least mine) keeps growing as I go longer and longer with disproportionate slot allocations. IOG have told me before that there is no configuration issues with my pool so something has to give and for the good of all small pools IOG should at least give me more than the standard answer - “get more stake and you will get more slots” - that is not a fair answer as my stake should have returned ~ 14 to 15 slot allocations instead of the 3 that I have been given. Yes I could go away retire my pool and forget about 2.5 years of hard work supporting the eco system but if IOG are the responsible organisation that they claim to be they should at least give me 5 minutes of their time.

1 Like

As an update to this IOG have agreed to investigate this further after the Christmas holidays - hopefully we can find out for sure what may be going on.

1 Like

Your sense of frustration stems from the “black box” nature of the cardano-node software in the sense that you can’t verify it’s operation. If you could check things better yourself it wouldn’t be such a “black box”.

If you are in direct contact with IOG: You should ask them to provide a simple tool that outputs the leader VRF value when given the epoch nonce, pool key, and slot number. You could then run this yourself and compare it’s output to your relative stake proportion to see if your pool is leader for a particular epoch slot.

Such a simple tool would give you some ability to manually check things. For example, you could run the tool on some data where you were awarded slot leader and then compare with other values. This might give you some sense of assurance that things are working properly if you could see the actual output.

If I had enough Haskell ability to write such a tool, I would write it.

1 Like

Thanks for your help - yes I will mention this to them as suggested. I have ran the cncli query

cardano-cli query leadership-schedule
–mainnet
etc etc

every 5 days and it always shows empty. Would your suggestion give me more information? Once again thanks for your help