Hi,
I’m trying to figure out wether running the topology updater script (using the 3rd party server by https://api.clio.one/htopology) is a must,
or is it just a best practice, but my node will be able to run perfectly ok with a static topology file (even the one with only the default relays-new.cardano-mainnet.iohk.io peer in it) ?
I can’t find a clear explanation of this topology updater mechanism in the official Cardano documentations, hence would appreciate your help.
Thanks!
You need incoming connections to your relays from other relays in order to propagate the blocks minted by your block producer.
This will change with p2p, but until then… topology updater script or connections manually set by other SPOs to your relays.
Hi!
Running topology updater is an optional and probably is recommended way to do this.
Nevertheless you will be totally fine without it and there are several pool operators who don’t use it due different reasons.
I had discussion in github regarding this with @rdlrt . He explained me that this is not a mandatory thing:
What is mandatory is having an incoming peer who is indirectly fetching blocks from another block producer. Until P2P is live, IOG uses the relays specified in stake pool registration to extract and connect to peers (twice per day). topologyUpdater provides a method for community to discover other live peers without a specific preference - you can read more about it here.
“Totally fine” is debatable. You only depend on one incoming connection to distribute your blocks as fast as possible to the whole blockchain, instead of having tens of incoming connections. I find it very risky.
got it. so it is not mandatory per se, but it helps propagate my produced blocks faster to the entire network, which is obviously important.
Thank you all!
Here’s a long, related discussion.
I myself use the updater. In almost all examples the block producer IP is provided to the remote service as a query parameter, but that is not required if you manually add it to the topology file after pulling topology data.
I read that discussion link and I would summarise it as follows:
The reason for 2 (and 3):
Unfortunately some people who run very small pools may not be setting up their nodes well. They think it is unnecessary to remain online much since mostly they don’t produce blocks. They just keep running their leader-logs to see if they get awarded a block.
Then when they get awarded a block, and just before they need to mint it, they make sure their block-producer is running and pulling blocks from lots of other nodes. However, they now don’t have many incoming connections because they haven’t been remaining on the network much before this time and they haven’t been pinging information to api.clio.one or similar.
Consequently when they do produce their block, its propagation through the network is slow.
This results in the next block-producer not seeing their block before producing theirs. Now we have two blocks with the same block number and conflicting transactions propagating across the network. Only 1 of these blocks will be adopted and thus rewarded with staking rewards.
Unfortunately, the second block-producer is likely at a disadvantage because this stake pool is more likely to be a bigger pool. The nodes adjudicate which block to adopt according to the VRF calculation based on keys and stake distribution. This puts the small pool at a massive advantage because its VRF score is likely to be very low (since it has little stake).
Thus the second pool gets penalised for the poor running and disconnectedness of the small pool.
For more information see this link:
P2P cannot come soon enough on mainnet!
While it is not mandatory your relay needs some other relays propagating your minted blocks to other relays. Otherwise, you won’t receive rewards from your blocks because the won’t appear in the blockchain. Having said that - the only way is to connect your relays to relays from other pools that run topology updater instead of your relays.
If you feel uncomfortable with the shell script there’s an alternative written in python. And - IOG has an alternative called p2p where relays manage to find other relays to connect to themselves. However, p2p is not officially released and should not be used at the moment.
I guess we all look forward to start using p2p as it also reduces the restarts of relays and increases throughput of transactions.
@jf3110, you are a bit wrong here.
Your block is propagated by fetching them, if you submitted your pool certificate with correct relays, then at least IOHK relays will be able to connect to your relays and fetch blocks.
It is not ideal and I personally do use topologyupdater, but this doesn’t mean that you can’t mint block without it.
Sure you can mint your block as this has nothing to do with how many incoming connections you have. However, if you have few incoming connections then your block will propagate slowly. This results in the next stake pool operator minting his block without receiving yours first and thus there are now two conflicting blocks.
The one accepted by the nodes is the one with the lower VRF value which is more likely to be the smaller stake pool operator. Unfortunately some small stake pool operators switch their nodes on/off and don’t maintain good network connectivity and this means that tools like “topology updater” don’t list their relays in the topology files given out.
I do tend to agree that topology updater is recommended way to do this, I just want to be clear and transparent and that everybody knew that running topology updater is not something mandatory.
I agree that it is more desirable to have more good incoming connections then less, but again if you have just some small amount of very good connections, then maybe your block propagation times will not be too slow. This is pure speculation and we can debate all day long about this, but my point is that, topology updater is optional and you should be able to mint block and propagate them to network without huge problems.
P.S. I by myself do use topology updater and I do recommend other to use it.
What I would like to know is how necessary it is to ping api.clio.one every hour?
I assume that if I don’t have my relays do this then my relay IPs won’t be provided to other operators when they download their topology files. Thus resulting in my relays having less incoming connections.
That is correct, if you don’t do this every hour, your relays will be removed from that list (probably not immediately) and they will not be provided to other relays downloading a list of relays, so your number of incoming connections will decrease.
I never said, that a block cannot be fetched for propagation. It is just, that there has to be some relay to propagate your block. Whether your relay connects through registering with topology updater or through some other mechanism does not matter. However, it is important, that a network of relays is maintained. That’s what topology updater provides and if there are other relays that pull your block, they have to either use topology updater to be part of the network or have further relays that propagate blocks which then use topology updater.
I know this is not an ideal solution, but it’s the way it is at the moment. p2p will solve it.
This is wrong. If you registered your relays correctly then you will get incoming connections from IOHK
Well, there’s an easy way to figure out. If you don’t have somebody to connect to your relay and if you don’t register with topology updater, you won’t have any incoming peers. That’s a fact.
The only other way some relays would discover to yours would be p2p relays currently testing on mainnet. I, personally, would not want to challenge luck for this to happen.
And as you said - you’re using topology updater. If you believe you won’t need it then why?
Again you are wrong here. You will have incoming peers, at least from IOHK, they get peers from your registration.
I don’t say I don’t believe in topology updater, I’m using it. I’m just trying to explain how things work.
Well it seems that since I am a small pool operator I have little to lose. I can just stop pinging api.clio.one and see how many incoming connections I end up with.
It seems that I won’t be penalised if my blocks propagate slowly but rather other pool operators will be punished for my disconnectedness.
Great!
There are no incoming peers from IOHK - the only exception would be from p2p testing.
If you don’t have any blocks scheduled, then you can try. Other pool operators will only see the effect, if they don’t update their topology and restart relays. However, from experience, there exist quite some of those. They will also be the ones still connecting to your relay.
A simpler approach would be to register a new pool and never run topology updater. Since this happens from time to time with new operators (you can track them here in the forum), none of them got incoming peers from other pools.
It’s quite reasonable the IOHK ran connecting relays at the beginning of Shelley era to keep network connected. As of today - with about 3000 pools active, those pools can organize themselves and through topology updater.
Anyways, with the activation of p2p all of that discussion will become pointless, because p2p is the better and decentralized solution.