Announcing Helm Charts to Run Cardano nodes in Kubernetes

Hi Everyone,

I’m thrilled :star_struck: to release production ready Helm Charts to operate Cardano nodes in Kubernetes. I put a ton of security and container :whale: experience to release the best possible Helm charts ever!

Here you go: find it on my Github

:four_leaf_clover: :four_leaf_clover: :four_leaf_clover:

Let me know your feedback

Cheers, Seb.

6 Likes

Woah. This looks awesome. Nice work @regel. Can’t wait to try out.

3 Likes

Wow great idea it will make stake pool deployment easier. We’ll definitely look at your charts

3 Likes

Unfortunately, I can’t comment (thread locked?) in 47906

I would be interested to get your feedback @tomdx Thomas, in particular in the area of peer-to-peer topology updates. I’ve been incubating something new in this area similar to your initial idea to use on-chain data in order to find registered relays information.

1 Like

Currently I deploy my cardano services with docker compose. Although I have K8S scripts, I don’t use them because they’d create an unnecessary overhead, especially on the RaspberryPi4. On Helm charts, I can’t comment unfortunately - never used them.

1 Like

Hey, I’ve done some work here in the past, see my repo.

I’ve also have seen this repo.

I would be open to combining efforts and having a single repository that can be community maintained. SPOs are going to play bigger roles in the upcoming infrastructure updates and I would like to add support for Mithril when it’s ready.

github/cardano-community is sadly a user’s account.

I have github/cardano-spos as an organization. Let me know if anyone is interested!!!

1 Like

Hey, many thanks for the links. I will have a look :eyes:

You can submit contributions, they are welcome!

Ps: I can’t move things to an organisation repo at this time. I need a public portfolio of things I’m doing and this Azure contribution falls in this category.

1 Like

More news :newspaper_roll: I have just updated the Helm Chart to use the Alonzo version (docker image tag 1.30.1).

Wait, there is more. Now you can find a Terraform folder. It makes it super simple to roll-out all required resources at once in Azure!

You can get a full-node deployed in five minutes on the testnet. Final steps, such as loading keys in the Azure Key Vault, and registering the node on the blockchain shall still be done manually for security reasons.

I’m still not pleased with the time (hours!) it takes to sync the Cardano blockchain initially. My next step is to improve this significantly. I want terraform apply to create a full node from scratch and synced at 100% in less than ten minutes, either on testnet or mainnet.

I ran some tests today, :cool: now the whole process takes 11 minutes to get a brand new Cardano cluster running in AKS and fully synced with the (testnet) blockchain. :100:

Very useful for testing and cost-saving since Cardano clusters can now be created in minutes with a single command (terraform apply -auto-approve), and destroyed in minutes with a single command too!

Updates in the git repo will be coming soon. :partying_face: with a new Helm Chart version.

Output of kubectl after 11 minutes:

$ kubectl exec -ti -n testnet testnet-cardano-relay-0 -c node -- cardano-cli query tip --testnet-magic 1097911063
{
    "epoch": 172,
    "hash": "9c140ab6e8f984edb0c758541edb2ef0ed6af5b03905816856bbb95159e349df",
    "slot": 44024468,
    "block": 3118583,
    "era": "Alonzo",
    "syncProgress": "100.00"
}

11 minutes, bring your own keys, and you’re done!

1 Like

Awesome job, I will give a try soon in this chart, thanks for sharing it.

2 Likes

It now takes 20 minutes to create a brand new full node on mainnet, fully synchronized (epoch 306) with the blockchain. This is the best I manage to get. If there is room for improvement :1st_place_medal:, let me know!

Here is how the 20 minutes are spent:

  • 7 minutes in the pod Init container to download and extract a 14GB mainnet tar.gz file (it’s 22 GB uncompressed, so my PV in Azure is 64 GB in size)
  • 13 minutes in the cardano-node pod container to open the ‘immutable db’
  • few seconds to open the other dbs and then start fetching new blocks from other peers

Stay tuned, I will clean up the code and commit to helm-charts repo tomorrow night :first_quarter_moon:

Helm chart (version 0.1.3) and Terraform samples published on Github :star:

3 Likes

fyi: I cleaned up Terraform files and made re-usable Terraform sub-modules. It’s here. Star :star2: the repo: GitHub - regel/terraform-azure-cardano: A Terraform Module for how to run Cardano on Azure using Terraform

Hello all, what will be the next step :man_shrugging: this year :question:

I let the Cardano community decide. Let me know your thoughts and vote below :arrow_down:

  • Work on existing / new tools to make key management easier.
  • Work on Terraform sub-modules for other Clouds (aws, gcp, …).
  • Work on a grafana dashboard (useful resources available and that can be reused?)
  • Work on a new cardano-cli query pools command. Yeah, we actually need this extension in the cardano-node core to make p2p efficient.
  • Something else entirely, add a wish below.

0 voters

1 Like

Terraform is an interesting twist to extend ease of deploy beyond helm charts. The idea of abstracting away infrastructure requirements for Cardano relay/passive node deployments within cloud services could allow more decentralization and easier management of node pools for dapp development so we aren’t all relying on blockfrost as a bottleneck! It goes hand-in hand with what you were already working on previously. I think anyone who subscribes to the k8s kool-aid or at least knows the basics like me would be able to figure out how to use KMS or other solutions to key management easily enough.

Hey @DinoDude thanks! To follow-up on your feedback:

I opened 93828 to discuss the dApp development process and understand the basic requirements.

I also opened 93830 to start a discussion on decentralisation and the basic stuff that appears to be missing in Cardano. I believe we, as a community, must have high expectations on decentralisation and raise our concerns if things are not headed in the right direction.

Hey everyone, I am glad to finally be able to release the Cardano Helm Charts version 0.3.0 on Github!

Star the repo :star:, share, and sponsor me please! :four_leaf_clover:

Changelog for this release:

  • Ogmios is now a first-class citizen in the cardano relay node(s).
  • P2P improvement, solving item 3 in the previous vote. See discussions in 93830
  • CI runs Checkov security scan and solved most warnings.
  • Clean up Redis pub/sub and upgrade dependencies.
  • Clean up documentation and added useful command lines to installation NOTES
  • Tested on mainnet and testnet, both with and without an initial blockchain tar.gz file (see persistence.sourceFile.url documentation).
  • I increased the default PVC size for mainnet to 128 GB, since the current blockchain size is 36 GB on mainnet and it is growing fast.

Hey. Do you have any intention of updating this or moving it to a community repository to allow shared ownership?

hello @beardedgeek and welcome, thank you for this first post.

will I update it?

Yes. Waiting for the 1.35 dust to settle and for the Vasil fork to complete. Docker images published lately by IOHK have been very confusing and I do not want to get lost in this.

moving it to community repo?

I included the CONTRIBUTING.md file in the git repository. Follow contributions guidelines and submit pull-requests if you want to. I will make sure to test and do the code review. I will also attend the next Cardano event in Lausanne, CH, and look forward to discussions that will happen there.

1 Like

hey, happy new year and merry Christmas :christmas_tree: :gift: :gift: :gift: to all Kubernetes and Cardano fans! :smiley:

I have excellent, and more excellent news: I updated the Helm charts to support the latest Cardano image along with simplifications and extra. Credits to @_2072 for the memory optimization tips! :four_leaf_clover: (avg. memory for mainnet relay pod 8.5G, same for the producer pod).

With this release, you can get a testnet (preprod) node fully synced in 3 minutes from scratch, mainnet fully synced in 15 minutes. Preview testnet is also an option, via a simple configuration flag.

Love, stars, and support much appreciated!

2 Likes