Provide high quality multiarch docker images and k8s support

Developers
1

Area
Docker/Kubernetes

Describe the feature you’d like

I suggest a series of improvements and bug fixes in order to provide a high quality Cardano Docker image. In the not too distant future it may become a reality that Cardano nodes need to get integrated in corporate IT infrastructure. They may become destinations or sources for various corporate data streams and as such, it’d be good to have an offering that allows these organizations to integrate Cardano with already existing cloud infrastructure.

With Kubernetes as the de-facto standard for container deployment, orchestration, monitoring, scaling , etc, it should be as easy as this to integrate a Cardano node …

kubectl apply -f https://raw.githubusercontent.com/input-output-hk/cardano-node/master/cardano-node.yaml

This issue suggests the following improvements …

  • [#2361] Added support for arm64 and perhaps for other target platforms to follow
  • [#2353] Support for graceful shutdown in container environments
  • [#2362] Self contained topology updates (until there is native p2p support)
  • [#2363] Deployment scripts for Kubernetes and perhaps also for Docker Compose
  • [#2364] Comprehensive documentation on how to run Cardano in managed container environments

Describe alternatives you’ve considered

An alternative would be to have advanced image functionality be provided by 3rd party and let this repository continue to focus on bare metal installations and more or less leave the existing (x86_64) docker image as is.

IMHO, it’d however be much better to have this tech be provided upstream.

Additional context / screenshots

As far as I can tell, much of the above is low hanging fruit and if it’s ok with you, I’d volunteer to do this over the course of the following weeks/months. The multiarch bit may initially require a workaround until we have the regular nix build that works on arm64. The image cannot rely on an external process to do topology updates - this has to be done within. Initially we could perhaps use guild-operators for that (to be discussed). Kubernetes support and getting started docs should not be a big deal.

cheers
– thomas

11 Likes
2

Here is a status update …

PR [#2393] is the third installment of [#2360]

So far we have …

  • Preserved functionality via -e NETWORK
  • Node configuration through env vars, which is later needed for k8s integration
  • Integrated configurable topology update
  • Graceful shutdown on docker stop
  • Improved access to the CLI

There is a README about this here.

1 Like
3

I really appreciate all the work you are doing for this and it has helped me a great deal with my raspberry pi stake pool setup, I really believe docker and k8s support is needed!

I would possibly think about working with the whole community on this by making this “providing high-quality multiarch docker image and k8s support” a proposal for project catalyst fund4 under the Dev Ecosystem challenge. https://cardano.ideascale.com/a/campaign-home/25868

I am a Community Advisor and would like to see something like this in the fund4 challenge and would like to assist in any way I can.

2 Likes
4

It is almost done already. Issue [#2360] and PR [#2393] haven’t seen feedback from the core folks yet - so I don’t really know whether/when this is going upstream.

The big blocker is [#2391], which unfortunately I can’t fix myself. It currently prevents the build for the arm64 image.

For nessusio/cardano I found a workaround that doesn’t use Nix, but I guess this is not a viable solution for the upstream build.

5

I will work on the issues you have linked, but I am wondering if you have thought about making this whole K8s/docker/multi-architecture project to the Project Catalysts community?

6

I have seen some other proposals about Docker, but none of them have done the work you have or even mention multi-architecture support for things like arm64

7

I doubt there is enough meat to it to become an independent Catalyst project. For now, I’d be happy to take care of this topic for as long as needed. I applied to the “small pool funding” thing.

8

I personally do think this has merit for at least some community funding, I have voted for projects with fewer working solutions, but regardless I will be following your work and will contribute in any way I can.

9

I don’t know about Nix build though on RPi4, haven’t tried yet. I will try tomorrow.

10

You will likely hit that brick wall - on the Pi as well as on a beefy AWS arm64 machine.

1 Like
12

Hi Tom, I have linked you github/gitbook to the Arming Cardano telegram channel Telegram: Join Group Chat, we are a channel of Stake pool operators running on rpis. Many of us have experience that may be beneficial. I want to THANK YOU again for everything you provide to Cardano dev community!!!

13

Status update …

For now, PR [#2456] is the fivth and last installment for this CIP

Now we have …

  • Preserved functionality via -e NETWORK
  • Node configuration through env vars, which is later needed for k8s integration
  • Deployment scripts for Kubernetes and Docker Compose
  • Integrated configurable topology update
  • Graceful shutdown on docker stop
  • Improved access to the CLI
  • Comprehensive documentation for Kubernetes & Compose

There is a README about this over here and a more verbose and user friendly GitBook over here.

PS: Once this is merged we’d need to cleanup a few links and perhaps find a better home for the docs.

2 Likes
14

Hey @tomdx I have only just come across all your hard work on this. I have been super busy with client projects and not had enough time to keep up to track. I also started a similar track of work, but you hare much future along. I will run through all your documentation and provide some feedback / updates.

When I have this working and in a stable condition I am happy to port to helm and GitOps with examples.

Thanks for this.
Cheers

15

@wIvie happy to let you know that I’ll follow your advice and submit a catalyst proposal. They are asking me for a referral email - would you have one for me?

image
image1582×720 58 KB

16

I am SUPER happy to hear this!

Yes, please use: wael.t.ivie@gmail.com

I am actually using your image right now :wink:

Do you have a pool id or website?

I will add it to my proposal on https://cardano.ideascale.com/a/dtd/ARMing-Cardano/340480-48088 catalyst as well to bring you some recognition as well.

17

Great, thanks. ASTOR is linked to my profile
Ideascale is also asking for a payment address (see above) - I think they ask for that of the referral (i.e. yours instead of mine)

18

I’ve been watching this for a little while, and have implemented some basic Helm charts to deploy a Cardano cluster based on the nessusio/cardano-node container images.

It fires up a 3 relay + 1 producer cluster. It supports importing your keys as well. Still purely experimental, but it works. It is still very much a work in progress.

2 Likes
19

Hey @Bruce_Lee_Harrison thanks for the reference :slight_smile:

ASTOR is now paying dividends for loyalty.

20

Hi Thomas, thanks for sharing.
I’ve already setup my kubernetes nodes, and now I’m still confuse to choose the docker image from IOHK or from your nessus-cardano. The link

https://raw.githubusercontent.com/input-output-hk/cardano-node/master/cardano-node.yaml

is invalid. Do you have another yaml file either for IOHK image or from your image? In the Github Repo from GitHub - tdiesler/nessus-cardano: A Cardano playground that explores various build/runtime aspects of the project. Something like an incubation space, before we are propose changes upstream. The initial focus is on "container first" for the Cardano node. I cannot find any yaml file for the deployment.

Cheers,
Yogi

21

The PR has been waiting on IOHK for ages, so you won’t get it from them for now. The k8s stuff inthe incubator is here