You are wrong again, I know several pool operators who run pools without topology updater and produce blocks successfully.
For example this pool:
@os11k: The issue is not whether the blocks will be produced OK.
The issue is whether the blocks produced will be propagated quickly.
I never said, that you can’t produce blocks and that you’ll have to run topology updater. Please don’t repeat that false statement.
All you need is at least one relay pulling your block into the network. The point is, to successfully maintain the network of relays there’s a centrally maintained list where you can register and get a list of other relays. If you use that service is up to you. If you don’t you’ll have to have some other relay(s) pulling your block - and most likely that one registered through topology updater, because it needs some other relays propagating your block into the network.
And - as has been said again and again - there’s some relays testing p2p - which obviously don’t need topology updater - that connect to relays registered on the blockchain.
If you can’t understand that I can’t help you.
It takes about 4 hours after you stop pinging api.clio.one to get of their list of registered relays. Usually SPO’s setup their systems to retrieve a list of relays every 24 hours (some every 12 hours).
I.e., you would start having less peers connected after about 28 hours. And, from my experience there’s some SPOs which don’t restart their relays that often. So, if you got some of them, they’ll still connecting to your relay.
Ok, sorry if I misunderstood.
I’m happy that we are on same page. TopologyUpdater is not mandatory thing to be, mandatory is to have incoming peers. Per my knowledge you will get incoming peers without topology updater.
Not sure what we are arguing about 
Cheers!
1 Like
I think the confusion is because the “topology updater” script/s are used for two things.
- Used to pull a topology file from api.clio.one. You can use this file to tell your relay which other relays to connect out to. In other words, this list determines your half-duplex connections OUT. These connections allow your relay to pull blocks from other relays.
This first use is optional. You can set up the peers you want your relay to connect to (mainnet-topology.json) however you want. api.clio.one just makes it easy to get a geographically well distributed list of relays.
- Used to tell api.clio.one that your relay is online and working properly. These are the “pings” every hour that most people set a cron job to run. api.clio.one collates this ping information from all relays that ping its service. api.clio.one collates this list of relays and gives out portions of this list to each other relay who chooses to download topology files from it. IE. the relays that also do the previous option (1.). This is how other relays then setup their half-duplex connections IN to your relay. These connections allow other relays to pull blocks from your relay. Any blocks your block-producer produces will be pulled by your relays from your block-producer. Then other relays on the cardano network need to pull these blocks from your relay to the rest of the cardano network.
This second use is somewhat required at present. If you don’t do this pinging of api.clio.one then other relays won’t know to connect to your relay and thus you won’t get incoming connections. If you don’t have these half-duplex incoming connections then no-one will pull your created blocks from your relay to the rest of the cardano network.
Now, if you list all your relays in your certificate file when registering your stake pool then this detail is on the blockchain. In this case maybe 1 or 2 IOHK relays will try to connect to these listed relays. I am not sure how much you can rely on that though. It is also possible that api.clio.one attempts to connect to all blockchain listed relays even if they don’t issues pings to it. I also don’t know about that. That would need to be clarified with IOHK and the operators of api.clio.one.
1 Like
That’s the way it is. A work around would be to ask some fellow SPOs to connect to your relays and pull your blocks. These fellow SPOs then use topology updater (the ping part) to get their relays connected. In that way your relays just live behind those fellow relays.
While it is true that relays can be registered via the certificate, quite a lot of them aren’t. But the real point is, that neither IOHK nor clio.one (which happens to just a SPO like all of the others) connect to relays without the hourly ping. The one exception is p2p network tests, because p2p uses the registered relays for discovery.
As I’ve said: if you don’t like the shell script - I have developed a variant in python with added functionality (automatic monitoring) and a simple to maintain JSON file for configuration. It works the same way and I’m using since its creation.
That is the important bit I wasn’t sure about. So you definitely need to continue pinging api.clio.one or you will not have incoming connections to pull your created blocks. Causing propagation delays for your block. Sure, you can use whatever script you like to do this (I have my own), but you must do the pinging, even if you are a small pool and don’t have blocks to mint.
It is a real pity that the punishment for not doing this is dished out to the well connected, well run pool that produces the next block without seeing your block first.
There’s an important exception - pool operators can agree on each other to build their own network. You can decide on your own how many and which relays your static configuration contains. I.e., not only your own relays. They just need at least one relay to connect to the global network (and of course the global network to theirs). Is this a bad thing? No, not necessarily. Relays and connectivity should be balanced and performance is important.
That’s why we all look forward for p2p to go into production. It’s an important step into more decentralization of the network. It will also improve block propagation a lot. And, with faster propagation, parameters like block size can be increased in safer way.
1 Like
OK, so let’s think things through a bit:
Imaging a group of stake pool operators get together and set up a service called “poolweapon.io”. You don’t need to have an account to use “poolweapon.io” but you can create an account by sending some Ada to obtain a key… But, only if “you want to support their service”.
“poolweapon.io” is a fantastic service because it allows everyone to easily download their mainnet-topology.json file. Anyone can download a specifically tailored topology file, they don’t even need an account. “poolweapon.io” is very successful and greatly appreciated by the community. The vast majority of stake pool operators are using it.
However, “poolweapon.io” has a dark secret. They greatly preference account holders by supplying only account holder relays to anyone downloading topology files using their service.
Consider how the cardano networking stack works where connections are half-duplex and blocks are only pulled and cannot be pushed. This means there are many “pull” connections from relays owned by “poolweapon.io” account holder members but almost no “pull” connections from relays owned by non-members.
Now imagine there is a chain fork (2 competing blocks which are both valid) where one block was made by a “poolweapon.io” member and one made by an outsider. Let’s call these 2 forks the “member” fork and the “outsider” fork. Member blocks get pulled quickly by others, whereas outsider blocks get delayed because no-one that uses the “poolweapon.io” service is pulling these blocks.
If the next block produced is made by a member then it will also get pulled quickly and added quickly to the “member” fork. Whereas outsider created blocks are added more slowly and are more likely to result in forks upon forks. The result is that the member fork grows more quickly.
Eventually, the Nakamoto consensus mechanism used by Cardano will approve the longest chain. This means that most chain splits result in favour of the “member” fork.
This is great for the members of “poolweapon.io” because they proportionally earn more staking rewards than non-members.
Yes, sure. How many operators will actually add extra connections to “non-member” relays? How do they even know who the “non-members” are?
I would say YES, this indeed is a very bad thing!
Ps.
Please note: I am not suggesting that any service is actually preferencing particular members over others. I am just saying that I have an issue with relying on such services because trust will be betrayed eventually.
Furthermore, organising a “clique” for swapping relay information between clique members is not an appropriate solution.
There’s only one problem with that scenario: there is no benefit, because SPOs playing foul won’t receive rewards and loose their delegations.
Pools are scheduled to validate a block by algorithm and not by competition. Pools not properly participating won’t get their blocks into the network and thus not receiving rewards.
Or in other words - there is no advantage in manipulating topology files, because you can either disconnect from the network (by excluding all connections to the network) and thus stop receiving rewards - or maintain connection to the network. That would make no difference to the current setup.
It is not the SPOs that are playing foul. It is the “poolweapon.io” service that is playing foul. It can create two separate classes of people: members and outsiders.
The topology configuration tells your node who to pull from. You cannot push blocks. You are dependent on other operators including your relay in their topology configs.
If “poolweapon.io” service does the following:
- Provides members with lists containing only member relays
- Provides outsiders with lists containing a mix of member relays and some outsider relays
So members will pull blocks only from members. Outsiders will pull blocks from both members and outsiders. Thus a net flow of blocks from members to outsiders but not from outsiders to members.
As far as incoming connections are concerned:
- Members have incoming connections from both members and outsiders
- Outsiders have incoming connections only from other outsiders
Now consider that the members are the larger group with more stake and therefore more block production.
Now a chain fork happens where a member produced one fork and an outsider the other. Members will keep adding their blocks to the member fork but they will not add to the outsider fork because they do not see outsider blocks. Thus the member fork will grow quicker since they are the larger group with more block production.
In this scenario, the algorithm used to determine the accepted chain is: Nakamoto consensus - longest chain wins.
The “poolweapon.io” members benefit because the described setup will tend to cause forks when outsiders produce blocks and the member version of the fork will be adopted. Thus members get a higher proportion of the rewards than they should. Outsiders get less.
The problem is it doesn’t matter how well you set up your outgoing connections or how well you run your pool. You still can’t control who sets up incoming connections to your relay.
Currently, we are all relying on a service like “poolweapon.io” and it has this potential to be abused.
It does not work that way. Because you can either isolate members from the network, and with that they no longer participate and won’t get rewards. Or if the members participate they’ll also pull blocks from outsiders and distribute them in their network. You can’t have it both ways.
The members are the majority, so of course they are “participating in the network”.
The outsiders are effectively segregated from the network except they don’t know it. They remain sync’ed because they are receiving blocks from insiders.
They can check that they have some incoming connections, but they don’t know who the IP addresses belong to (insiders or outsiders). If all their incoming connections are from outsiders only, then their blocks will not propagate properly to the network of insiders. Consequently their blocks become temporary forks on their versions of the chain.
Eventually even the outsiders nodes adopt the insider forks (longest chain rule) because the insiders keep only building on their forks and the insiders are in the majority.
The key point is that insiders have incoming connections from both insiders and outsiders so their blocks flow to the entire network. However, outsiders only have incoming connections from outsiders so their blocks remain only with outsiders. None of the insiders pull blocks from outsiders.
If everyone becomes dependent on a service like “poolweapon.io” then it has the power to set up this scenario.
One can implement that without such a “service” just by firewall rules.
But yes, it is possible to lockout a pool via social effort. It’s just that if only one pool opens the door for those outsider’s the whole effort won’t work. Which is quite likely the case.
As I’ve said, one can configure a set of static nodes to connect to in addition to the downloaded topology. And I know a couple of operators which connect their relays through static configuration. That way your outsider’s would get their blocks pulled, thus working around the fence you described.
And of course everybody can implement an alternative service which does not exclude other pools. I guess that would happen in if your scenario would become reality.
And again - with enabling p2p - central topology distribution will be history.
Yes. If one or more of the insiders pulls blocks from the outsiders then the scheme described above will fail.
The problem is that no-one knows how a service like api.clio.one distributes its lists of relays to downloaders. They say they use geography and randomise the selection. Their service is dominant with vast majority of operators using them.
From what I can tell, api.clio.one seems to be doing a good and honest job. My blocks do seem to propagate reasonably quickly. Although maybe I could be an insider?
However, I set my relays to pull from 20 other relays in my topology config, but I only have 6 or 7 incoming connections sometimes and there never seems to be more than 12 incoming.
Do most pool operators configure their relays to have 20 outgoing connections? If so then I should have closer to 20 incoming connections on average if api.clio.one somewhat randomises the selection. (And I do run relays in different parts of the world.)
This is what made me think about how a dominantly used opaque service like this could be abused by insiders.
We must remember that for our blocks to be pulled we are dependent on other relays being configured to connect in to us. Services like api.clio.one dominantly control this configuration at present.
P2P can’t come soon enough.
Default value is 15, but one can change the value - and of course there’s your static configuration, basically your other relays and the block producer.
I have about the same amount of in-peers for all of my relays and never had a problem with minting blocks.
If you fell uneasy with your situation, you can ask for some static additional connections.
And yes, p2p is the better solution. It also enables bi-directional communication. But before it should be enabled some serious testing is required. Because if it does create separated network rings it would be even worse. I’ve seen test results where p2p improves block propagation big time. It is as key feature for further block size increasing.