While encouraging decentralisation on a software level is great, it’s not useful if everyone uses the same hardware providers in the same locations. For example, if a majority of the Cardano Nodes are located in one country and that country is compromised in some way, such as a natural disaster, then that is a weak link in the network.
As it stands, there are very few fully operated Cardano pools in Australia. There are some that have relays in Australia, but the block producers are not in Australia. That is why we feel it is important that our pools are physically in Australia to bring true decentralisation and security to Cardano.
By operating pools in Australia, we aim to bring security and stability to the Cardano network. By being located in Australia, Australians and nearby countries will also experience faster transactions on the Cardano network.
[AOAUS] ADA Ocean Australia:
5b9a5633289c52003d359bce18f77f571a701fd3942a52bde3262558
Low Fees:
Fixed Cost: 340 ₳
Margin: 1%
High Uptime:
Operating System live patches, virtual machine live migration and auto start are just a few things we use to make sure our servers always stay up and running smoothly.
Thanks! At this stage there isn’t many places to promote pools within the Cardano community without the threat of being banned. It does make it extremely difficult to promote a pool.
Please take a look at my site… stakingforgood.com
It is still a work in progress but it is intended to be a directory for pools donating a percentage of pool fees to charitable organizations!
Your point about relay and block-producing nodes being in different places is a good one. I personally don’t do it—CRAB’s nodes are all in Germany (which coincidentally seems very popular)—but I wonder about the effect of high latency on block submission to the chain… Is anyone aware of any analysis or explanation about this effect?
I also wonder why some stake pools are placing relays and nodes in different countries anyway; surely it’s advantageous to be as near to your block-producing nodes as possible, rather than hiding a latency effect within the internal network. I’ve also noticed some stake pools are using DNS round-robin to balance over multiple countries; that makes the reported country of the relay inaccurate, and multiple attempts at connecting could yield very different latency results.
@ tiredpixel
From a strategic point of view it makes sense that a relay and block producer should be in the same general location. Splitting them up for “redundancy” makes no sense because if the block producer goes down, all the relays have no point. As in, your block producer is the “weak link”. So if it’s in the same datacenter, it’s all in the same fault tolerance compared to having relay nodes across the world.
You could have a block producer on standby in another location with the server certificate on it, but it only runs as a relay node. If the block producer stops working (such as a datacenter goes down), then the other relay could be turned into a block producer until the other comes back up. It’s a very manual process though and the original block producer would have to be known that it would be going down for a long time for this to been a solution.
Although I agree with most of your first paragraph about the strategic point of view, I’d like to observe that there is arguably a difference between benefit to the network as a whole (which does not get directly rewarded), and benefit to the stake pool operator (which does get rewarded—in theory, at least; if you’re a very small pool with an expected average of blocks significantly << 1, then this doesn’t apply). Having relays distributed, even between countries, helps the network, so splitting them up does technically help. However, it doesn’t directly reward the operator. That’s why, of course, we generally talk about relays and block-producing nodes together, since the incentive to run a relay is to have it as part of the private network with a block-producing node available. In such a (normal, I imagine) scenario, I would expect the optimal behaviour for the stake pool operator to be to have minimal latency between the block-producing node and its relays (although perhaps a single low-latency link would be enough, since it would arrive and propagate first? worth thinking about, perhaps).
What I do with CRAB is similar to what you suggest, except fully-automated. Block producers are available on standby, and manage themselves to form a quorum, and a secondary is promoted when needed. In fact, this switchover happens within a few seconds, which leaves the recovery startup typically slightly higher than the boot time needed for Cardano Node. Since CRAB is still on 1.18.0 (as 1.18.1 doesn’t compile and then got pulled, and 1.19.0 doesn’t compile either), this boot time is large (a few minutes), but I have high hopes for once we’re able to upgrade to 1.19 or later.
The article was interesting and well-written; thanks for that. I would certainly agree with the overall premise of multi-country, and indeed multi-continent, decentralisation being important. I myself would consider hosting a pool elsewhere, except I already have extensive infrastructure within highly-available datacentres in Germany, which I developed for Isoxya. Hence, it makes most sense for me to use spare capacity within my existing network to run CRAB. This might change in the future, but almost certainly not whilst the expected return from running the pool is zero, since everything I’m running is at a loss.
I was thinking about that and I have come to the same conclusions, so I pretty much agree with everything you said. Although I think in terms of the relay and low latency, I’m not sure it does have an impact as the network elects a block producer. As long as it’s online and synced, then it should produce the block and submit it back to the relay nodes which submits to the network.
With your implementation of HA, that sounds pretty much the same as the way MongoDB operates isn’t it (three nodes that hold an election to decide who’s running)? If I understood your explanation. If so, that’s what I was considered in trying to implement. Based on the conversation going on in Redit, it appears others are also doing it. It’s a pity it’s not part of the Cardano framework yet.
My node is at 1.19 and as long as it’s fully synced, killing the node and starting it up and running happens in seconds. So yeah, once you get that building correctly it’s going to be a sweet setup.
I know what you mean. I’ve started the nodes up from scratch, no existing servers. I’m definitely going to be running at a loss for a while.
Interesting what you point about about the network electing a block producer; perhaps latency doesn’t matter, after all. I’d be interested in hearing more about this from someone who understands the network algorithms better than I do.
Yes, exactly—very similar to MongoDB. A built-in cluster capability for Cardano Node would be a very nice addition, presuming that it was simple and reliable enough to use other the external alternatives I already have set up.
Ah, that sounds great about 1.19; mine still takes minutes to reboot each time. I’ve been given advice last night about cherry-picking onto 1.19.0 in order to fix builds, so I’ll try to give that a try soon.
Yeah, I chose to expand my base infrastructure at around the same time, so there was definitely a jump in my cost, but that wasn’t purely because of CRAB pool, given the other projects I work on. So I’m basically using spare capacity, but it’s not like it’s free, either—and that’s before you count the literally weeks of full-time work I’ve put into it researching, developing, and testing everything. We’ll see.
That’s what we’re hoping to achieve. As it stands more than 50% of the Cardano network are controlled within the U.S and Germany. So from a software point of view Cardano is going great, but once more countries join in then it will decentralise more in a physical sense.
Mainland? Not sure what you mean with that. The idea is if Cardano was at full capacity and a country drops out, you want to be able to “route” to other countries until it’s fixed. As it stands there isn’t much traffic on the network so there isn’t much impact, but long term there could be.
Yes, a great answer to the question, caseygibson, thank you. I noticed that even now the first movements on exchange platforms do not start, what would it mean? Has the Cardano not returned to its normal functioning so far?
When I think of “natural disasters” - I live in Florida, so hurricanes, flooding, and since we’re the lightning capital of the US, power outtages come to mind - I think of literally natural disasters when I hear that, but another form of disaster that’s definitely often nationwide in scope are political upheavals and wars. Look at Hong Kong now, under the ‘thumb’ of the CCP, so much for crypto progress there - and the rampant runaway hyperinflation in Venezuela.
Anyway, best of luck on developing your staking pool business.
Also, I wouldn’t worry too much about governments getting rid of cryptos, the younger generations are generally pro-crypto on both sides of the political spectrum, so as they come into power over the years, and the old, relatively ignorant politicians retire, conditions will improve for our industry, particicularly as our industry grows financially, they will lobby the government to support our industry, and they will have the influence to do it too.
Governments getting rid of cryptos would be like governments getting rid of cell phones - they will only be taking a step back in time as the rest of the world moves ahead with this technology.
