That’s what I’m saying - the same 1.35.3 node missed 2 blocks when using new KES keys + op cert (generated on 1.35.3).
After restoring the old KES keys + op cert generated using 1.34.1, the very same node with the same configuration minted a block.
Hello @AF.PXLZ
Smaug Pool tweeted that “–consensus tpraos” should be used instead of praos. Maybe that is the issue you are having? (Just a guess) 
Source:
I don’t think the cardano-cli version is the problem. But in your case, I would generate new KES keys and a new operational certificate now with cardano-cli 1.34.1, paying attention to the current kes interval and the certificate counter, and then replace the ones that will expire in 2 days.
Hey there! I’m not using cncli with my build so I don’t think that this is the issue. Thanks for the link though!
This is the current experiment I’m running. Generated new KES keys + op. cert via cardano-cli v. 1.34.1. Got a block incoming in a couple of days, I’ll revert back with the results.
Just out of curiosity @georgem1976 when you run the query kes-period-info command on your 1.35.3 build, does the qKesKesKeyExpiry output null or does it output an actual date?
This is on my primary BP, upgraded to 1.35.3, but the kes keys and the operational certificate were generated with 1.34.1 about 4 weeks ago.
✓ Operational certificate's KES period is within the correct KES period interval
✓ The operational certificate counter agrees with the node protocol state counter
{
"qKesCurrentKesPeriod": 539,
"qKesEndKesInterval": 562,
"qKesKesKeyExpiry": null,
"qKesMaxKESEvolutions": 62,
"qKesNodeStateOperationalCertificateNumber": 8,
"qKesOnDiskOperationalCertificateNumber": 8,
"qKesRemainingSlotsInKesPeriod": 2978293,
"qKesSlotsPerKesPeriod": 129600,
"qKesStartKesInterval": 500
}
cardano-cli 1.35.3 - linux-x86_64 - ghc-8.10
git rev 950c4e222086fed5ca53564e642434ce9307b0b9
4 weeks ago, just after replacing them, the output was (with cardano-cli 1.34.1):
$ cardano-cli query kes-period-info --op-cert-file node.cert --mainnet
✓ The operational certificate counter agrees with the node protocol state counter
✓ Operational certificate's kes period is within the correct KES period interval
{
"qKesNodeStateOperationalCertificateNumber": 7,
"qKesCurrentKesPeriod": 500,
"qKesOnDiskOperationalCertificateNumber": 8,
"qKesRemainingSlotsInKesPeriod": 7477679,
"qKesMaxKESEvolutions": 62,
"qKesKesKeyExpiry": "2022-09-22T21:44:51Z",
"qKesEndKesInterval": 558,
"qKesStartKesInterval": 496,
"qKesSlotsPerKesPeriod": 129600
}
I get this (both on 1.35.3):
KES + op.cert that are about to expire:
✓ Operational certificate's KES period is within the correct KES period interval
✓ The operational certificate counter agrees with the node protocol state counter
{
"qKesCurrentKesPeriod": 539,
"qKesEndKesInterval": 540,
"qKesKesKeyExpiry": "2022-08-26T21:44:51Z",
"qKesMaxKESEvolutions": 62,
"qKesNodeStateOperationalCertificateNumber": 8,
"qKesOnDiskOperationalCertificateNumber": 8,
"qKesRemainingSlotsInKesPeriod": 126228,
"qKesSlotsPerKesPeriod": 129600,
"qKesStartKesInterval": 478
}
Newly rotated KES + op cert:
✓ Operational certificate's KES period is within the correct KES period interval
✓ The operational certificate counter agrees with the node protocol state counter
{
"qKesCurrentKesPeriod": 539,
"qKesEndKesInterval": 600,
"qKesKesKeyExpiry": null,
"qKesMaxKESEvolutions": 62,
"qKesNodeStateOperationalCertificateNumber": 8,
"qKesOnDiskOperationalCertificateNumber": 9,
"qKesRemainingSlotsInKesPeriod": 7902103,
"qKesSlotsPerKesPeriod": 129600,
"qKesStartKesInterval": 538
}
Not sure why the qKesKesKeyExpiry outputs as null though I’ve seen it pasted around on the IOHK github as well.
Maybe qKesKesKeyExpiry is a new option for after the Vasil Hard Fork.
It should output the date that your KES expires as it does in your 1.34.1 output no?
I don’t know what should be displayed there with 1.35.3… But my pool is minting blocks (3 this epoch so far, all adopted).
Yeah so the issue could have to do with KES keys that have been generated more recently or IDK. I added this as a bug on the cardano-node github. Hope they get to reviewing it and revert.
I am running 1.35.3 and I get the same as @georgem1976 :
CARDANO_NODE_SOCKET_PATH='/run/cardano/mainnet-node.socket' cardano-cli query kes-period-info --mainnet --op-cert-file /etc/cardano/private/node.cert
_ Operational certificate's KES period is within the correct KES period interval
_ The operational certificate counter agrees with the node protocol state counter
{
"qKesCurrentKesPeriod": 539,
"qKesEndKesInterval": 562,
"qKesKesKeyExpiry": null,
"qKesMaxKESEvolutions": 62,
"qKesNodeStateOperationalCertificateNumber": 2,
"qKesOnDiskOperationalCertificateNumber": 2,
"qKesRemainingSlotsInKesPeriod": 2969823,
"qKesSlotsPerKesPeriod": 129600,
"qKesStartKesInterval": 500
}
My kes keys and operational cert were generated with node version 1.34.1 some weeks ago.
You might get faster response on this, and possible solutions, if you post this question/issue out in the open on Twitter… then tag @adamKDean and @IOHK_Charles. 
It’s been a day since your post and no useful response yet. You might be wasting more blocks.
Thanks but I think we learned the hard way Twitter is not a space for discussing dev issues. The issue was also reported on IOHK’s github. As stated above, I am already trying some fixes and will revert back with results. EOD I will probably go for a fresh install. Cheers!
All things considered, I think it produced a positive outcome. At least from my perspective. Sure, the fudsters had a field day about it. But it also forced key decision makers to pay attention. Anyway, just my thoughts.
I think we need to hear if anyone else out there has minted blocks on 1.35.3 while running a new op.cert / KES keys generated with cardano-cli v 1.35.3.
Right now I’m running a 1.35.3 node with a 1.34.1 op.cert/KES, but it expires in 7 days. Hopefully we’ll get more clarity over the next few days as to whether this is a common issue to op.certs/keys created with cardano-cli 1.35.3?
-Sully at RADAR
1 Like
Currently using 1.34.1 newly generated KES keys + op certificate and waiting for the scheduled block to see if it goes through or not. Will revert with results.
2 Likes
I’ve seen that IOG people (Kevin and Carlos) have already responded.
Hopefully, it will get resolved soon.
I have also recently updated my Opcert with 1.34.1 before upgrading to 1.35.3
I also see:
qKesKesKeyExpiry": null,`
qKesNodeStateOperationalCertificateNumber": 1,
qKesOnDiskOperationalCertificateNumber": 2,
It might say “null” since the new cert has not minted a block and registered the new opcert on chain?
I have a block scheduled in the coming days, hopefully without any issues - I will update you following the scheduled mint
At least 1 person has:
https://twitter.com/GeanBrinker/status/1563148677405097986
Same case here… that’s why I’m also eager to confirm this.
1 Like