Does cncli database store correct leader_vrf values?

Terminada · 28 September 2022 13:03

I noticed a couple of slot battles shown on pooltool recently and was trying to see the leader vrf values to confirm how they were resolved. I pulled the data from my cncli database and found that the leader_vrf_0 values were not always less for the winner. But, I thought that the winner should have the lower leader vrf value according to this reference:

github.com/input-output-hk/ouroboros-network

Consensus should favor expected slot height to ward off delay attack

opened 05:12PM - 30 Jan 21 UTC

Straightpool

**Internal/External** *External* **Summary** When a pool produces or propa…gates a block late so the block collides with the block of the next slot leader, only the vrf value is evaluated to determine the winning block, which is on its own the correct strategy deciding randomly between competitive slots. Due to the current logic in the case of delayed blocks it does happen that the block of the next slot leader which was properly propagated and produced on-time is lost due to the misconfiguration of the prior slot leader. This can be seen as a form of attack from the viewpoint of the on-time pool. Similarily, a later slot leader could produce his block multiple seconds earlier and collide with the previous block, if his vrf value was lower he could attack the previous block leader as his early block would make it on chain, the on-time block of the prior slot leader would be lost. We do not see this type of attack yet, as this would be a conscious effort, right now this attack is most likely without malice just out of misconfiguration. **Steps to reproduce** Steps to reproduce the behavior: 1. Wait on a situation where there are two slots with only a few seconds "x" apart 2. Delay production of first block by "x" seconds on first slot leader 3. Produce second block on second slot leader on-time 4. Wait until the block of the first slot leader has the lower vrf value 5. Observe that the block of the first slot leader makes it on chain, the block of the second slot leader is lost (had both blocks be on-time, both blocks would have made it on chain) **Expected behavior** The consensus protocol should evaluate the slot of the blocks and favor the block group which is expected in the current time frame. With *expected* I refer to the exact block slot height. The algorithm can calculate precisely which slot# a block at this exact moment in time should have. If there is more than one block in that group of "on-time" blocks only then the lower vrf should decide the winner. The block of the pool which produced the block on-time and propagated the block swiflty should not be attackable by a prior slot leader who delays his blocks accidently or on purpose or by a following block leader who produces his block multiple seconds earlier by modifying the system time on purpose as we have seen on the ITN as a tactic to win competitve slots. **System info (please complete the following information):** - OS: Ubunto - Version 20.04 LTS - Node version: cardano-node 1.25.1 - linux-x86_64 - ghc-8.10 git rev 9a7331cce5e8bc0ea9c6bfa1c28773f4c5a7000f **Screenshots and attachments** ![2021-01-30 17 57 49](https://user-images.githubusercontent.com/42584250/106362733-b3b6b480-6324-11eb-809b-7f6dd1d6e45b.jpg) See epoch 244: https://pooltool.io/pool/000006d97fd0415d2dafdbb8b782717a3d3ff32f865792b8df7ddd00/orphans This is the propagation delay of the slot leader before my block: ![2021-01-30 17 59 21](https://user-images.githubusercontent.com/42584250/106362778-f4aec900-6324-11eb-8ad0-8f42d449b946.jpg) See propagation delays of the pool before my block here: https://pooltool.io/pool/59d12b7a426724961607014aacea1e584f3ebc1196948f42a10893bc/blocks This is the hash of the winning late block which made it on chain: ca40eed5fd46f76fbf64e17a98808f098363a83dfe8c100046947505baa1e406 My block made it into the orphan list on pooltool, hash: 97abb258f15995688bdacdc75a054883b22471451026f409a967028ec7b30316 This is a log excerpt from my block producer, the block which should have been the parent for my block arrived full 4 seconds late: {"at":"2021-01-28T07:16:47.00Z","env":"1.24.2:400d1","ns":["cardano.node.ChainDB"],"data":{"kind":"TraceAddBlockEvent.AddedToCurrentChain","newtip":"97abb258f15995688bdacdc75a054883b22471451026f409a967028ec7b30316@20251916"},"app":[],"msg":"","pid":"582044","loc":null,"host":"foobar","sev":"Notice","thread":"49"} {"at":"2021-01-28T07:16:48.04Z","env":"1.24.2:400d1","ns":["cardano.node.ChainDB"],"data":{"kind":"TraceAddBlockEvent.SwitchedToAFork","newtip":"ca40eed5fd46f76fbf64e17a98808f098363a83dfe8c100046947505baa1e406@20251913"},"app":[],"msg":"","pid":"582044","loc":null,"host":"foobar","sev":"Notice","thread":"49"} **This is the 2nd time I have observed this, last time was on December 21st, same pattern different slot leader:** Block producer log. {"at":"2020-12-20T03:07:09.01Z","env":"1.24.2:400d1","ns":["cardano.node.ChainDB"],"data":{"kind":"TraceAddBlockEvent.AddedToCurrentChain","newtip":"78f0c4a29a9c2b9a628584066f05ba3285f6b7eaac3bc270e353f52a0fa94a8c@16867338"},"app":[],"msg":"","pid":"582044","loc":null,"host":"foobat","sev":"Notice","thread":"49"} {"at":"2020-12-20T03:07:10.64Z","env":"1.24.2:400d1","ns":["cardano.node.ChainDB"],"data":{"kind":"TraceAddBlockEvent.SwitchedToAFork","newtip":"2c237fded6c534200814d991deccc3c99f0a1bae01e603e743d6d5926e8a4519@16867333"},"app":[],"msg":"","pid":"582044","loc":null,"host":"foobar","sev":"Notice","thread":"49"} 78f0c4a29a9c2b9a628584066f05ba3285f6b7eaac3bc270e353f52a0fa94a8c was my block which was orphaned 2c237fded6c534200814d991deccc3c99f0a1bae01e603e743d6d5926e8a4519 was the hash of the block before mine (5 slots before) arriving 6 seconds late. Mike downloaded the json of one of the blocks of the pool before mine and noticed a delay of about 10 seconds back then: {"height": 5100112, "slot": 16870897, "theoretical": 1608437188000, "tiptiming": [10547, 10416, 10440, 10509, 10350, 10099, 10432, 10428, 10333, 10378, 10427, 10548, 10219, 10111, 10362, 10293, 10350, 10281, 10296, 10410, 10461, 10419, 10484, 10343, 10350, 10485, 10347, 10330, 10530, 10592, 10327, 10290, 10373, 10332, 10192, 10288, 10390, 10375, 10392, 10301, 10369, 10457, 10350, 10439, 10354, 10493, 10323, 10503, 10407, 10337, 10343, 10398, 10442, 10359, 10367, 10325, 10334, 10305, 10499, 10369, 10346, 10231, 10369, 10311, 10317, 10420, 10505, 10303, 10240, 10310, 10560, 10350, 10360, 11098, 10410, 10310, 10310, 10280, 10320, 10563, 10370, 10330, 10280, 10120, 10400, 10310, 10350, 10310, 10340, 10490, 10460, 10380, 10540, 10410, 10340, -1608437188000, 10330, 10290, 10340, 10370, 10420, 10310, 10260, 10320, 10380, 10440, 10380, 10370, 10350, 10420, 10270, 10517, 10560, 10360, 10110, 10410, 10380, 10300, 10420, 10440, 10390, 10640, 10580, 10580, 10550, 10280, 10740, 10400, 10580, 10380, 10380, 10420, 10380, 10400, 10320, 10370, 10360, 10450, 10300, 10500, 10340, 10410, 10320, 10300, 10550, 10360, 10410, 10320, 10350, 10400, 10350, 10240, 10630, 10370, 10457, 10350, 10330, 10340, 10530, 10280, 10320, 10737, 10310, 10300, 11560, 10479, 10360, 10290, 10430, 10380, 10280, 10360, 10330, 10410, 10310, 10380, 10320, 10320, 11710, 10320, 10310, 10340, 25580, 10450, 10400, 10320, 10440, 11766, 10390, 10310, 12846, 10320, 10320, 12740, 12500, 12952, 13053, 18000, 20610, 20610, 24800], "histogram": "[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

Here is a query that lists the 5 most recent slot battles where the winner had the higher leader_vrf_0 result stored in the cncli database (when the winner should have the lower value):

sqlite3 /var/lib/cardano/cncli.db 'select winner.block_number, winner.slot_number, winner.pool_id, winner.hash, winner.leader_vrf_0, loser.leader_vrf_0 from chain winner inner join chain loser on winner.slot_number = loser.slot_number and winner.block_number = loser.block_number and winner.orphaned=0 and loser.orphaned=1 and winner.leader_vrf_0 > loser.leader_vrf_0 order by winner.block_number desc limit 5;'

BlockNo | SlotNo | Winner PoolID | Winner block hash | Winner Leader VRF | Loser Leader VRF

7814905|72793921|ac5ba151fae6889e25021acc723b0e34655828bf3328b6f1180e1d65|fd15a900f46e8bf9c03ad987da1d99e9102dea99ae1e0423113ba5c7ad1f903b|0003b0ad7390b0da547e3e43b1adae072c709d5bd06d4962ef992ac42221deb3|00016073b18d7c4f9ae4174085401da8426eae37646a29c0027b0ba57ee8f456
7814595|72787725|ee98a72dfb3ba43e9de2e39ef825a69d5d45627d2e918ea3e260f2dd|7b02eb92d0f5ac4e820d7535f18825d242086336413a6c51e30a1c9d581594d2|0004106510affa94afddb43845a59dc6041b3887ce563e7f1114018f75329072|0000d4662f2c7d626857c552b6063b2c70f891b2d602412a6a15baa2fd200a43
7814482|72785510|b619d366a0acc2e5a8e0f8e373d6eb926a01811f05a381aaa7a8564c|99656c88d38916a8ce932d53a5a06db8ebbafb0ff1a37112ea86c248a03539c1|000703cbe5d02357b7dcd72b6319679ac58089eb91a8ff9a961e3050d41ec0fb|00024b9090d39771158f8426e876f687053c0efe2fe4f8335b34f656006183b5
7814446|72784819|971212db48d9390662faa0fa725950f3b9ac1a898227f5fcf1096399|c97725be2f3854730037af89ace0f6a4e1dabedf70040b56842b87795c26f2e3|00039b61db164c09c820bf86194d5b8e58f2c5400f6f6ec8f017ec397389a343|0003307de189e839e1a5ee43dd37553f505383f3b1407071b316a8ac63426c90
7814351|72782882|ed40b0a319f639a70b1e2a4de00f112c4f7b7d4849f0abd25c4336a4|dc67f5747cfbe163ef6efd7a29e5e49296504757372bdb8fa7575e65eb79ae26|0008cdbb377d1a589908b8a763f95a3878ccc916110dd5575fcdb89094d13698|00003e02943eba2de8de080a022878350b26d1a0bff472c6b370496519b9d722

If you take the top example. The leader_vrf_0 value for the winner was 0003b…, but the loser had a lower value of 00016…

Pooltool shows this battle and confirms the winner block has hash fd15… which is the same as my query confirms was the winner : Cardano PoolTool - The most comprehensive staking statistics for Cardano on the web.

According to the reference, the lower leader vrf value should win the slot battle.

The format of the chain table in the cncli database is:

sqlite3 /var/lib/cardano/cncli.db '.schema chain'

CREATE TABLE chain (
id INTEGER PRIMARY KEY AUTOINCREMENT,
block_number INTEGER NOT NULL,
slot_number INTEGER NOT NULL,
hash TEXT NOT NULL,
prev_hash TEXT NOT NULL,
eta_v TEXT NOT NULL,
node_vkey TEXT NOT NULL,
node_vrf_vkey TEXT NOT NULL,
eta_vrf_0 TEXT NOT NULL,
eta_vrf_1 TEXT NOT NULL,
leader_vrf_0 TEXT NOT NULL,
leader_vrf_1 TEXT NOT NULL,
block_size INTEGER NOT NULL,
block_body_hash TEXT NOT NULL,
pool_opcert TEXT NOT NULL,
unknown_0 INTEGER NOT NULL,
unknown_1 INTEGER NOT NULL,
unknown_2 TEXT NOT NULL,
protocol_major_version INTEGER NOT NULL,
protocol_minor_version INTEGER NOT NULL,
orphaned INTEGER NOT NULL DEFAULT 0 ,
pool_id TEXT NOT NULL DEFAULT ‘’,
block_vrf_0 TEXT NOT NULL DEFAULT ‘’,
block_vrf_1 TEXT NOT NULL DEFAULT ‘’
);
CREATE INDEX idx_chain_slot_number ON chain(slot_number);
CREATE INDEX idx_chain_orphaned ON chain(orphaned);
CREATE INDEX idx_chain_hash ON chain(hash);
CREATE INDEX idx_chain_block_number ON chain(block_number);
CREATE INDEX idx_chain_node_vkey ON chain(node_vkey);
CREATE INDEX idx_chain_pool_id ON chain(pool_id);

Are the leader_vrf_0 values that cncli database stores not the proper leader vrf values?

The leader_vrf_1 values are empty and I don’t know what eta_vrf_0 or eta_vrf_1 values represent.

Can anyone explain this?

Is there some other way to look up what the leader vrf values were for each block in a slot battle? The system logs don’t show these values and nor does pooltool when it shows the slot battle winner.

HeptaSean · 28 September 2022 20:26

Interesting question!

Tried to understand it from https://github.com/cardano-community/cncli/blob/develop/src/nodeclient/sync.rs, but couldn’t.

Do block_vrf_0 or block_vrf_1 make more sense?

Terminada · 29 September 2022 01:00

Indeed it is interesting!

I just did some more investigating and this is starting to look like a change (or bug) introduced more recently.

Here is what I have found so far:

I did a query to retrieve all “slot battles” where the winner did not have the lower leader vrf value (cncli database leader_vrf_0). I noticed that these were very infrequent before the last few epochs (when everyone started running node version 1.35.3).

So I started at the earliest examples and looked through my node logs. The earliest such example was 2022, July 09. As I worked forwards in time, I found that in each case the “slot battle” turned out to be a “height battle”. What happened in each case that I checked so far is that my node did initially prefer the block that was ultimately orphaned switching tip to it (because it had a lower leader vrf). But, it subsequently received another block that forced it to switch tip again shortly after.

So it appears that what actually happened in these early cases (before node version 1.35.3) is the following:

Node A and Node B both produce blocks for same slot
Node A initially wins the slot battle (because it has a lower leader vrf) and nodes that received both blocks switch tip to node A’s block
Node C produces a block on top of node B’s losing block (presumably because it didn’t see node A’s block yet)
Now the combination of blocks created by node B and node C are the longest chain
All other nodes upon seeing node C’s block then switch their tips to this longer fork

So the above is exactly how I thought things should work. Slot battles are resolved by lowest leader vrf and height battles are resolved by longest chain. Sometimes a slot battle can later become a height battle if the next producer didn’t see the other block prior to making its block. This is exactly what Duncan Coutts outlined as the current state at: Consensus should favor expected slot height to ward off delay attack · Issue #2913 · input-output-hk/ouroboros-network · GitHub

Here is an example of one such slot battle that subsequently became a height battle.

Jul 11 18:35:04 relay1 cardano-node[962057]: [relay1:cardano.node.ChainDB:Info:143] [2022-07-11 08:35:04.87 UTC] Valid candidate ea82dc57bda01a664b6aa8591d788458b885baef2c685b9ca9fb5d9495e0acdd at slot 65962213
Jul 11 18:35:04 relay1 cardano-node[962057]: [relay1:cardano.node.ChainDB:Notice:143] [2022-07-11 08:35:04.87 UTC] Chain extended, new tip: ea82dc57bda01a664b6aa8591d788458b885baef2c685b9ca9fb5d9495e0acdd at slot 65962213
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.ChainDB:Info:143] [2022-07-11 08:35:05.26 UTC] Block fits onto some fork: 90a921ca2c93cb69cf52fe5d2cc532df7990d38a344e3db3daf22457dba6a370 at slot 65962213
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:05.27 UTC] before next, messages elided = 38
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:05.27 UTC] [TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:2700, remoteAddress = 192.168.27.9:2700}) (Right [At (Block {blockPointSlot = SlotNo 65962213, blockPointHash = 90a921ca2c93cb69cf52fe5d2cc532df7990d38a344e3db>
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.ChainDB:Notice:143] [2022-07-11 08:35:05.29 UTC] Switched to a fork, new tip: 90a921ca2c93cb69cf52fe5d2cc532df7990d38a344e3db3daf22457dba6a370 at slot 65962213
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:05.81 UTC] [TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:2700, remoteAddress = 88.99.143.230:3001}) (Right [At (Block {blockPointSlot = SlotNo 65962214, blockPointHash = 36fc6ae41f912253084168b1582fb6beb97a8e8ce2fa05>
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:05.86 UTC] [TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:2700, remoteAddress = 52.59.248.215:4321}) (Right [At (Block {blockPointSlot = SlotNo 65962214, blockPointHash = 36fc6ae41f912253084168b1582fb6beb97a8e8ce2fa05>
Jul 11 18:35:05 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:05.87 UTC] [TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:2700, remoteAddress = 94.136.30.220:3001}) (Left (FetchDeclineConcurrencyLimit FetchModeDeadline 2)),TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:>
Jul 11 18:35:06 relay1 cardano-node[962057]: [relay1:cardano.node.ChainDB:Info:143] [2022-07-11 08:35:06.08 UTC] Block fits onto some fork: 36fc6ae41f912253084168b1582fb6beb97a8e8ce2fa05834159c9f865dfe30b at slot 65962214
Jul 11 18:35:06 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:06.08 UTC] before next, messages elided = 11
Jul 11 18:35:06 relay1 cardano-node[962057]: [relay1:cardano.node.BlockFetchDecision:Info:153] [2022-07-11 08:35:06.08 UTC] [TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:2700, remoteAddress = 54.148.179.11:3001}) (Left (FetchDeclineConcurrencyLimit FetchModeDeadline 2)),TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:>
Jul 11 18:35:06 relay1 cardano-node[962057]: [relay1:cardano.node.ChainDB:Notice:143] [2022-07-11 08:35:06.09 UTC] Switched to a fork, new tip: 36fc6ae41f912253084168b1582fb6beb97a8e8ce2fa05834159c9f865dfe30b at slot 65962214

Notice how my node did prefer the ultimately orphaned block initially and switched its tip to it, but then received a subsequent block which made it switch again.

I have reviewed several of the earliest examples retrieved by my query and they all fit this pattern. According to the Duncan Coutts reference above, this is the correct action.

But, this is not how things appear to be working now.

Starting from the most recent blocks retrieved by my query and reviewing logs at these times shows that the node seems to prefer the later received block. Here is an example:

Sep 29 04:50:57 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Notice:178] [2022-09-28 18:50:57.48 UTC] Chain extended, new tip: 552a6898460ccaa343437c8aad7ba9648c2f57bee06444dd8dbc6a1abe71cb01 at slot 72824766
Sep 29 04:50:57 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Info:178] [2022-09-28 18:50:57.87 UTC] Block fits onto some fork: f1bbac626bc5d6ad88fdf86a84d25d299cdb337a2bb9c8a3c2277bad244fc4d9 at slot 72824766
Sep 29 04:50:57 relay1 cardano-node[972972]: [relay1:cardano.node.BlockFetchDecision:Info:188] [2022-09-28 18:50:57.88 UTC] before next, messages elided = 16
Sep 29 04:50:57 relay1 cardano-node[972972]: [relay1:cardano.node.BlockFetchDecision:Info:188] [2022-09-28 18:50:57.88 UTC] [TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:2700, remoteAddress = 35.247.85.163:3001}) (Left (FetchDeclineConcurrencyLimit FetchModeDeadline 2)),TraceLabelPeer (ConnectionId {localAddress = 192.168.27.7:>
Sep 29 04:50:57 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Notice:178] [2022-09-28 18:50:57.91 UTC] Switched to a fork, new tip: f1bbac626bc5d6ad88fdf86a84d25d299cdb337a2bb9c8a3c2277bad244fc4d9 at slot 72824766

See how my node switched tip to the later received block. But this later received block did not have the lower leader_vrf_0 in the cncli database. I don’t know what made the node switch its tip to this later block. Maybe just because it was received later??? The logs following this just continue on with blocks built on top as usual.

So I continued reviewing the more recent slot battles and in every case I noticed that the later received block was preferred irrespective of the leader_vrf_0 value in cncli database. For example:

Sep 29 06:57:19 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Notice:178] [2022-09-28 20:57:19.54 UTC] Chain extended, new tip: 5396e4543b3ee688bcaced6f515a4e54f1b22a5fd1f612be88625ab7b491040d at slot 72832348
Sep 29 06:57:19 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Info:178] [2022-09-28 20:57:19.55 UTC] Block fits onto some fork: c6651e4413b482d82dcc50f24eaeec8fdd1792938d6fe2bb2c43e8ec970f7cfc at slot 72832348
Sep 29 06:57:19 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Notice:178] [2022-09-28 20:57:19.55 UTC] Switched to a fork, new tip: c6651e4413b482d82dcc50f24eaeec8fdd1792938d6fe2bb2c43e8ec970f7cfc at slot 72832348

In that case the later block did have a lower leader_vrf_0 score.

But another example:

Sep 29 08:58:08 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Info:178] [2022-09-28 22:58:08.54 UTC] Valid candidate 4089eac285b5a0170974f1d2aced7f1a4035e6a329d7569f1ef1c36068d08b5c at slot 72839597
Sep 29 08:58:08 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Notice:178] [2022-09-28 22:58:08.54 UTC] Chain extended, new tip: 4089eac285b5a0170974f1d2aced7f1a4035e6a329d7569f1ef1c36068d08b5c at slot 72839597
Sep 29 08:58:08 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Info:178] [2022-09-28 22:58:08.60 UTC] Block fits onto some fork: 1643b6b22508648d8170af937548d1edf645f450dda88ad32e8472119cf1d306 at slot 72839597
Sep 29 08:58:08 relay1 cardano-node[972972]: [relay1:cardano.node.ChainDB:Notice:178] [2022-09-28 22:58:08.65 UTC] Switched to a fork, new tip: 1643b6b22508648d8170af937548d1edf645f450dda88ad32e8472119cf1d306 at slot 72839597

In that case the later block had a higher leader_vrf_0 score in cncli database.

Why did my node immediately prefer this later received block for the same slot when its leader_vrf_0 score was higher?

One thing that might be going on here is that the cncli database value stored as leader_vrf_0 might be incorrect. I notice that the length of this value has changed to be half the size more recently.

If you do this query to list all slot battles where the block with the higher leader_vrf_0 value won:

sqlite3 /var/lib/cardano/cncli.db 'select winner.block_number, winner.slot_number, winner.pool_id, winner.hash, winner.leader_vrf_0, loser.leader_vrf_0 from chain winner inner join chain loser on winner.slot_number = loser.slot_number and winner.block_number = loser.block_number and winner.orphaned=0 and loser.orphaned=1 and winner.leader_vrf_0 > loser.leader_vrf_0 order by winner.block_number desc;'

You will see that the top (latest) results look like:

7817452|72845687|e645c034522f32522bf1c64a7f46128c562880abc26abe5f65f25a84|25f0f2262cd6d4fc6afdcbcb20643028745bc5035ab3897dc75f5d19e40039e6|0005e6d2b1c42a5bcc81758673ea3c6837a9369d1fc4af8191064c3685accd55|0000d7fc65939da69a675516637da68eca480f4d8df8128dc97282364ea4b059

But the bottom (earliest) results look like:

7473625|65745540|43bf2e2c7601bd6dfb38564fc2c48f5b1a773fdb4095dee412f52e50|1c42cd7e8d7bb007828f7480844185b0c20998f9c5eb8ba9b87b2a8d20a5c7c9|0002efabf7f6a5f5fb85cf410817e33a6ee09dcf6fc0d7b806dea6f9b07505cabb0d42462c2bff7bfa76831dfd9872f606e3534fc8fd1736c07466be7b469fb3|00014cc9956e4511fc7aba887c157c617e74b36e69a237507bf768d9d5c66c58cae30ca9d9b009978a714b7a830038f980016829eb78a916bc9818885a8d17ff

Note how the leader_vrf_0 values are much larger for the bottom (earliest) results.

Maybe cncli is now storing incorrect leader_vrf_0 values??? Or maybe the cardano-node is no longer considering the leader vrf value when deciding on slot battles???

I am not sure how to investigate this any further.

Can someone check my logic and do a bit of investigating on their machines because I don’t want to start making claims that there is a bug if I am just hopelessly confused.

I do recall IOHK saying something about being able to remove an extra vrf calculation in each block for an efficiency gain. Could it be something related to this?

Is there some way to pull the leader vrf values for blocks directly from the chain???

Terminada · 29 September 2022 11:05

I think it is a bug with cncli since the length of the hex string leader_vrf_0 values in the chain table has changed recently.

I submitted a bug with cncli: Incorrect leader_vrf_0 values in chain table? · Issue #19 · cardano-community/cncli · GitHub

brouwerQ · 29 September 2022 12:07

The lowest VRF value rule only applies on chains of the same length, if another pool already built on top of the block with the higher VRF value (because he didn’t receive the one with the lower value in time), that chain is already longer and will be used by the next slot leader (if he received all the blocks in time).

Normally this situation will only happen if the next block slot is very close to the one(s) with the slot or height battle and the propagation of the one with the lowest VRF value took longer to reach that BP.

I don’t know if this is the reason for what you saw, because that situation won’t happen that often with current propagation times I think, so I might be something else.

Terminada · 30 September 2022 04:37

It is looking like the cncli database leader_vrf_0 values are not the correct leader vrf values used by nodes to decide slot battles.

I looked at logs in two different relays on opposite sides of the world for the last couple of slot battles. The blocks were received only a few milliseconds apart by both relays but in different orders. In both cases the same block was preferred immediately. Presumably the nodes compared leader vrf scores to determine the winner. This is expected behaviour. However the cncli database leader_vrf_0 values were actually lower for the losing blocks.

I have reviewed logs of several similar cases too. Note that the slot battles I am considering are simple slot battles, not ones where a subsequent block arrived quickly which could have been built on one fork before seeing the other, which would cause the node to decide based on longest chain rule.

So what we can say is that the leader_vrf_0 values in cncli database do not correspond with the “leader vrf” values the nodes are using to determine slot battles.

Therefore either cncli is wrong or cardano-node is wrong. Querying the cncli database shows that the correlation between slot winner and lower leader_vrf_0 values is 50%. In other words, the cncli leader_vrf_0 values are not correlated with the slot battle winner.

Note that the cncli leader_vrf_0 values were the correct values prior to the Vasil upgrade.

@AndrewWestberg said he was going to look into it next week: Incorrect leader_vrf_0 values in chain table? · Issue #19 · cardano-community/cncli · GitHub

For those interested he also recorded this excellent video in June: Cardano: NerdOut - Babbage LeaderLogs - YouTube

Terminada · 1 October 2022 00:24

It turns out that the answer to my original question: “Does cncli database store correct leader_vrf values?” is YES. @AndrewWestberg 's cncli tool is storing the correct values.

Instead, there is a bug in cardano-node which has now been confirmed.

Slot battles are currently being determined based on the “block vrf” value rather than the “leader vrf” value. In essence this means that there is a 50% chance to win a slot battle now, whereas before smaller pools were much more likely to win.

Small pool operators should campaign for this bug to be fixed soon as they have now lost their slot battle advantage.

Terminada · 1 October 2022 00:26

github.com/input-output-hk/ouroboros-network

Leader VRF value no longer settling ties

opened 10:56PM - 30 Sep 22 UTC

JaredCorduan

I do not know whether or not we've found a bug or just discovered the consequenc…es of an intentional decision. --- The final tie breaker for "slot battles" is the leader VRF value: https://github.com/input-output-hk/ouroboros-network/blob/9249a70ed9e2365f3963e47cb31b4b1589bca8f6/ouroboros-consensus-protocol/src/Ouroboros/Consensus/Protocol/Praos/Common.hs#L62-L68 In the `TPraos` protocol (used prior to the Vasil HF), `csvLeaderVRF` was the leader VRF value. In the `Praos` protocol, however, `csvLeaderVRF` is being set to the single VRF value in the block header (prior to the range extension). **This removes a small advantage that small pools previously enjoyed.** Small pools are more likely to win this tie breaker, since by being a small pool they need a smaller leader VRF value in order to win the leader check. Using the the VRF value before the range extension is applied removes this small advantage. --- The Evidence: The view, `PraosChainSelectView`, is populated by the `BlockSupportsProtocol` class method `selectView`, which uses the `ProtocolHeaderSupportsProtocol` class method `pHeaderVRFValue` to set `csvLeaderVRF` in the view. * In `TPraos`, `pHeaderVRFValue` uses the leader VRF value: https://github.com/input-output-hk/ouroboros-network/blob/9249a70ed9e2365f3963e47cb31b4b1589bca8f6/ouroboros-consensus-shelley/src/Ouroboros/Consensus/Shelley/Protocol/TPraos.hs#L111 * In `Praos`, `pHeaderVRFValue` uses the raw header VRF value: https://github.com/input-output-hk/ouroboros-network/blob/9249a70ed9e2365f3963e47cb31b4b1589bca8f6/ouroboros-consensus-shelley/src/Ouroboros/Consensus/Shelley/Protocol/Praos.hs#L141 --- This was discover here: https://github.com/cardano-community/cncli/issues/19

Topic		Replies	Views
Orphaned block, three way Height Battle Operate a Stake Pool	4	420	30 October 2022
Cntools.sh The number of leaders is inconsistent. Why Operate a Stake Pool	2	380	10 April 2022
Understanding slot battles Education	1	751	5 August 2023
Problems with using VRF to decide fork battles General Discussions	8	534	2 October 2023
Cncli-leaderlogs accuracy Operate a Stake Pool	34	1587	27 February 2022

Does cncli database store correct leader_vrf values?

Related topics