I’d like my pool to send its allocated slots to PoolTool however since I don’t use CNCLI, I have to use some scripts such as the one papacarp made years ago.
The problem is that it uses Jörmungandr and I am not even sure what that is… Some kind of legacy Cardano node? Some kind of generic node to make other blockchains? I found the IOHK official page and github but it is not clear to me.
When I read the script to send allocated slots on papacarp’s GitHub, it doesn’t seem too complicated (https://github.com/papacarp/pooltool.io/blob/master/send_slots/shell/send_slots.sh)
I think it could be done using the leadership-schedule query instead of using Jörmungandr but I didn’t find any documentation on the json format PoolTool wants so I am a bit stuck.
I tried to find some other implementation of such script but the pages I found are usually not available anymore.
Can someone help me find/create a script to send slots to pooltool without using CNCLI or Jörmungandr?
The easiest way is to use cncli. There are some extensions to the papacarp script. And only cncli implements it AFAIK. I started writing a script, but gave up, because some chain information is needed, which cncli has access to.
Why not using cncli? It is stable and does not use a lot of resources.
I have a working senttip script, but it does not deliver data for the metrics pages on pooltool.
I agree the simple thing you are looking for should always be possible without having to use some big package of things you otherwise have no interest in. The original code, which does what you describe (or at least did), is here:
I still use this today… and it still works with PoolTool.io … after setting it up this way in 2020. I may have had to patch it a few times, so if it doesn’t work as posted on GitHub (generally because of JSON query responses or cardano-cli syntax changing over the years) then I’ll upload the one that I’m currently using here.
If it exists elsewhere in a patched form or is maintained elsewhere, I’d also be happy to hear about it from others who may know. (The proper thing to do would be upload my patches as a GitHub PR somehow but there seems to be a lack of demand for fixing this old stuff.)
@jf3110 : I would prefer not to use CNCLI because I want to control exactly what runs on my machines. Also I feel like I get more knowledge by using cardano node and cli directly without anything else. I could however check if I could use CNCLI only to send the slots and nothing else… That would be the last option though.
@COSDpool : Thanks but I think you are mixing the “sendmytip” script which I use too (without cncli) and the “send_slots” one.
I have never sent my allocated slots hashes to pooltool, but I used to send my block receipt delays. I do however understand why some delegators might like such information for checking up on how well their pool is being run by it’s operator(s).
Maybe a better option, which doesn’t rely on pooltool as a centralised intermediary, would be to post the data on your own website about your own pool. You could have a “check my operator performance” page. On this page you post a hash of your allocated slots just before each epoch starts. Then at the end of the epoch, you post the raw leader-log so people can easily check and confirm it’s hash correctly matches. This would also allow you to provide an explanation, or analysis of the cause, for any missed blocks right there alongside the data.
I have thought about doing this sort of thing for the past 2 years. I think it would be better if something like this became the standard. It is a much more decentralised way for people to check up on their pool’s performance.
If you really want to take this further, you could create a standard for the display of such data. For example, you could specify:
json format for the leader-log records
hash algorithm to be used for initial hash commitment
Optional extra data record for the pool certificate recording the website location where this data can be scraped
Then such data could be utilised by wallets and displayed to users at the time they are making their delegation decision. This would be a much more decentralised solution than simply gifting control over such valuable, and somewhat personal data, to an individual or company. For example, think about the fact that blocks can be missed for various reasons and not simply for reasons of poor performance.
Not that I have anything against pooltool. They provide many fantastic services and I greatly appreciate their existence.
I haven’t taken these ideas any further myself for two reasons:
It is difficult to explain to ordinary delegators why blocks are lost when forks occur and how/why the winning fork was chosen. They simply see you missed a block and think you failed. There is also an inbuilt bias in the protocol against more physically remote pools. Due to their increased delays in sending/receiving blocks they get more “fork battles” and consequently more lost blocks since they will lose half these battles. This is something that needs to be fixed at the protocol level otherwise we are all incentivised to run our block producer from a data centre in Germany (like what happened during the incentivised test net period).
You already can get a pretty good idea about how your pool operator is performing by simply looking at their “lifetime luck” percentage on sites like cexplorer.io. If it is close to 100% after a long time of operation then they must be doing something right. Just relabel this to “performance percentage”, even though we all know that over the short term luck will significantly impact the figure. But, over the long term, it is a pretty good gauge. (Except for the fact that physically remote pools are disadvantaged.)
I have written my own scripts for reformatting leader logs for my time zone, sending block delay data, and pinging api.clio.one about my relays:
The reason why I want to use PoolTool is that it did become some kind of standard. For example it is a requirement in the Minswap ADA delegation framework to use PoolTool to assess pool performance.
I think the biggest puzzle piece I need to make a simple shell script that sends the slots using cardano cli is a sample of the json that is sent to PoolTool and the format in which the slots have to be provided. That would be the content of $CURRENT_SLOTS in this script : https://github.com/papacarp/pooltool.io/blob/master/send_slots/shell/send_slots.sh
If someone can provide me this that would be great
thanks, you’re right. I’d thought there was a systemd equivalent of send_slots as well but I’m not seeing it around (anymore)… and since I’ve never sent slots (originally I felt this was a privacy issue for our BP) I don’t have a working version to refer you to. I’ll keep in mind your further request & let you know if I find anything.
I can’t figure out what is the “userid” that is supposed to be sent, everywhere it says something like “GET THIS FROM YOUR ACCOUNT PROFILE PAGE ON POOLTOOL WEBSITE” but on my profile page I have a nickname and no userid.
I tried to send the following data
{“currentepoch”:“425”,“poolid”:“8dcdf33740ee8e9da6e36337d875fb9222f5c8a1a315fda36886c615”,“genesispref”:“8e4d2a343f3dcf93”,“userid”:“Cassandra”,“assigned_slots”:“2”}
With userid being
-My nickname
-The pool api key
-The Zapier API key (on my profile page)
-The stake address I use to log in in both formats
I always get the following response
{“message”: “Internal server error”}
And if I don’t put a userid
{“success”: false, “message”: {“error”: “userid not found in json data”}}
In CNCLI, PoolTool data consist of the pool API key, pool id and pool ticker so I suppose the userid should be the pool api key…?
Can someone help me find what is this userid? Or maybe why PoolTool returns a server error when I send the data with the pool API key (since I guess that’s the userid I should be using)
I haven’t actually ever released such information about my pool for the reasons I stated above.
However, if you think through what is trying to be achieved: Pooltool is looking to provide some verification service about the “reliability” or “success rate” of a pool operator. IE: what percentage of their allocated blocks did they actually produce. Yet, pooltool doesn’t want to receive the actual leader-log data ahead of when the blocks get produced because that would expose the pool to possible denial of service attacks if attackers knew when blocks were due. IE: You need to keep your leader-logs private until AFTER the allocated slots have passed.
So the scheme goes like this:
You produce your leader-log before the epoch starts as usual.
You create a hash of this leader-log and upload this hash along with the number of slots you have been allocated just before the epoch starts.
You perform your duties as a pool operator and produce your blocks during the epoch.
At the end of the epoch, you reveal the raw leader-log file and then anyone can check it’s hash to ensure it is the same value as the hash you provided at the beginning of the epoch.
This proves that you didn’t delete a record from your leader-log file because you missed that block.
It is a neat system, but I don’t use it because it doesn’t explain any of the reasons for why you might have missed a block. For example, my pool is housed in Australia and consequently suffers block delays of just over 1 second usually. This means that I will get involved in “fork battles” roughly 3 times as often as a pool that is housed in a data centre in US or Europe which suffers less than 1 second block delays. Since the winner of each fork is essentially random, then I will lose half those battles. This means that my pool will on average get over 7% of its blocks orphaned as opposed to one of the centralised “data centre” pools that will only get an orphan rate of around 2.5%.
Most do not understand the significance of this problem because most pool operators live in USA / Europe. For example, I have received feedback from others that the solution to this problem is to simply move my block producer to a warehouse in Germany. They say it doesn’t matter where your pool is located, what matters is who controls it…
Perhaps we need an entity like Amazon to start manipulating the network traffic in their data centre so that variable block propagation delays result in advantaging their “approved” pools. Maybe then people will properly understand that decentralisation also means decentralisation of physical location, particular jurisdiction, particular govt, AND hardware / network ownership.
So, could encouraging the publication of such data to become an expected standard have unintended consequences???
I also agree with what @Terminada said about the stigma of simply missing a block not being helpful to either delegators or pool owners without some context about the circumstances… or what they might be getting in exchange for those losses suffered, e.g. improved decentralisation from the remotely hosted pool they are considering supporting.
After a 2nd look at this I think the only way @CassandraDeVries to get your answer to this specific question (a matter beyond my experience because sending “tip” only requires the API key & not any more privileged access) is to post as an issue here; I can see form this query that the maintainer has responded to these issues as late as 4 months ago:
… but even if he’s not responding to issues, some SPOs who monitor that category would likely have an answer for you: with probably at least one person doing what you have described.
Well I had a look into how cncli generates the message to send slots information to pooltool and came to the conclusion that it is not worth the effort unless you want to re-implement cncli. It sends historic slot information from the block chain, which has been stored in the cncli database.
Btw. I spent enough time reading cncli code to recognize that it does not do any harm to your setup. You don’t need to run the sync on your block producer node either. You can use it remotely or from a relay node.
Not that I want to shill cncli here, but if you want to send slot information to pooltool I do not see any alternative.
You might be right but I am kind of stubborn. I feel like there is a very simple solution waiting to be found and I just need to continue searching for it.
I say this because I feel like PoolTool doesn’t actually use the slots list but only the amount of slots/epoch. So the json could be extremely simple to build from the leadership-schedule query.
Also, even if I am wrong and the slot list is indeed required, as soon as someone provide me with an example of that list before being encrypted, I am confident it will be quite easy to build from the output of the leadership-schedule query and the rest of the shell script with the encryption is already available.
This solution would mean that in the end I would have one simple shell script that rely only on one cardano cli command (leadership-schedule). For me this is the cleanest solution and is worth some effort to achieve it.
Wow, ok, there is a lot of dialog here on this that I wasn’t aware of. I’d be happy to help get this running again without cncli if there is interest in it. I’m going to reply on the github tracker as its easier to deal with the intricacies there.
Well if you think so; I gave up when I found out that some unspecified information about previous slots is required. That would mean to store some information in some kind of database. Which cncli already does. A shortcut would be, to use the cncli database to extract that information. But then you’ll have to have to do the sync with cncli (like gLiveview.sh does). Which led me to the point to use the full set of feature of cncli.
You know sometimes there are no simpler solutions than the ones already on the hand.
I’ve developed my own version of sendtips (using python) and used it for some time. But it lacks getting the metrics updated on pooltool and your pool does not show up on the network health screen.
That seems strange to me. At the end of the day, the agenda is to:
Commit a hash of your leader-log along with the number of blocks you have been assigned, before the epoch starts.
Reveal your actual leader-log at the end of the epoch that matches your hash commitment.
Use blockchain data to confirm how your pool performed.
Perhaps pooltool also wants you to send (at the end of the epoch) the block number and block hash for each block you produced to make searching the block chain for the proof easier?
In any case, I don’t see any reason to save the entire blockchain block history to a database, which is what cncli does, if all you want is the block numbers and block hashes for just your pool’s blocks. You could for example, grab such data from your own systemd logs.
What data is cncli actually sending when you send your confirmation at the end of the epoch?
