Test-Setup for Relay and Producer with Docker

Staking & Delegation Setup a Stake Pool
1

Hello,
I am currently trying to setup my first stake-pool in the testnet.

I created an docker-compose-file with env-variables and downloaded the sample configuration files.

Starting a normal node and syncing to the testnet works without any issues and I also can connect to the specified port of the container. (I used the default port 3001) It was necessary to bind the cardano-node process in the container to 0.0.0.0 instead of 127.0.0.1 to make it work.

telnet to 127.0.0.1 with port 3001 results in an established connection. But when I start another container that runs on a different port and use the topology file to sync with my first node, is does not work.

The first log-messages of the cardano-node that should connect to my relay-node look good, but then, the connection closed because of a timeout:

[b846b81f:cardano.node.IpSubscription:Info:204] [2021-02-23 20:40:09.24 UTC] IPs: 0.0.0.0:0 [127.0.0.1:3001] Trying to connect to 127.0.0.1:3001
[b846b81f:cardano.node.IpSubscription:Info:268] [2021-02-23 20:40:09.24 UTC] IPs: 0.0.0.0:0 [127.0.0.1:3001] Connection Attempt Start, destination 127.0.0.1:3001
[b846b81f:cardano.node.IpSubscription:Notice:204] [2021-02-23 20:40:09.24 UTC] IPs: 0.0.0.0:0 [127.0.0.1:3001] Waiting 0.025s before attempting a new connection
[b846b81f:cardano.node.IpSubscription:Notice:268] [2021-02-23 20:40:09.24 UTC] IPs: 0.0.0.0:0 [127.0.0.1:3001] Connection Attempt End, destination 127.0.0.1:3001 outcome: ConnectSuccessLast
[b846b81f:cardano.node.ErrorPolicy:Warning:206] [2021-02-23 20:40:09.24 UTC] IP 127.0.0.1:37405 ErrorPolicySuspendPeer (Just (ApplicationExceptionTrace (MuxError MuxBearerClosed "<socket: 99> closed when reading data, waiting on next header True"))) 20s 20s
[b846b81f:cardano.node.IpSubscription:Error:268] [2021-02-23 20:43:09.25 UTC] IPs: 0.0.0.0:0 [127.0.0.1:3001] Application Exception: 127.0.0.1:3001 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply)
[b846b81f:cardano.node.IpSubscription:Info:268] [2021-02-23 20:43:09.25 UTC] IPs: 0.0.0.0:0 [127.0.0.1:3001] Closed socket to 127.0.0.1:3001
[b846b81f:cardano.node.ErrorPolicy:Notice:202] [2021-02-23 20:43:09.25 UTC] IP 127.0.0.1:3001 ErrorPolicySuspendConsumer (Just (ApplicationExceptionTrace ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (ShelleyMAEra 'Mary StandardCrypto)) ('[] *))))))) (ServerAgency TokNext TokMustReply))) 20s

My docker-compose files look like this:

version: "3.9"
services:
  node:
    image: "inputoutput/cardano-node:1.25.1"
    ports:
      - ${NODE_PORT}:3001
    volumes:
      - ${NODE_BASE_FOLDER}/data:/data
      - ${NODE_BASE_FOLDER}/configuration:/configuration
    tmpfs:
      - /ipc
    command: run --config /configuration/config.json --database-path /data/db --topology /configuration/topology.json --host-addr 0.0.0.0 --port 3001 --socket-path /ipc/node.socket

with an .env file that looks like this:

NODE_BASE_FOLDER="/cardano-nodes/node-testnet" 
NODE_PORT="3001"

Does anyone have an idea, what might be the error in my setup?

I wanted to attach the config-files for the cardano-node also, but this filetype is not allowed in the forum.

BR,
Matthias

1 Like
2

Hi!

I think 2 containers has its own network namespaces… so if you want to connect from 2nd container to the 1rst then you need to use your host machine IP address instead of the localhost.

3

In your topology you’d want to reference the block producer IP from the relay and vice versa (i.e. not 127.0.0.1). This may work with using container names and let compose do the dns resolution.

With a plain docker setup, I use docker network create --subnet=172.18.0.0/16 cardano and then assign IPs explicitly with --ip 172.18.0.10 for example.

In k8s the dns resolution by container name works for sure, but I’m not so familiar with compose.

1 Like
4

or use --network host which will use the host network in the containers - means you dont need to expose any ports since it will be opened on the host network

5

Your were both right. I totally missed the fact, that 127.0.0.1 in the container is usually not the same as on the host.
I have a working configuration now. I will do some additional tests, as it did behaved differntly as I expected. For example, I tried to use host network and still use 127.0.0.1 for the target node but it did not work. But it worked, as soon as I used the host ip address. I am not sure, why this is the case.

The main thing is: It is working. Thanks guys!

This is the docker-compose.yml file I ended up using:

version: "3.9"
services:
  node:
    image: "inputoutput/cardano-node:1.25.1"
    volumes:
      - ${NODE_BASE_FOLDER}/data:/data
      - ${NODE_BASE_FOLDER}/configuration:/configuration
    tmpfs:
      - /ipc
    network_mode: host
    command: run --config /configuration/config.json --database-path /data/db --topology /configuration/topology.json --host-addr 0.0.0.0 --port ${NODE_PORT} --socket-path /ipc/node.socket

.env-file for docker-compose:

NODE_BASE_FOLDER="/cardano-nodes/producer-testnet"
NODE_PORT="3010"

topology.json (for the producer node container):

{
  "Producers": [
    {
      "addr": "10.114.0.2",
      "port": 3001,
      "valency": 1
    }
  ]
}

BR,
Matthias

6

or use --network host

This is a bad advice, because it would also expose the BP to world. Instead, you want the Relay to be accessible from other peers and the BP only from the Relay

7

is problematic. Please make sure that the BP is only accessible from the Relay and not from the world

8

I would definitly protect it with firewall-rules.

Dont’t bind the cardanot-node to the external ip at all is definitly better. Up to now, I didn’t have a working solution and therefore wasn’t sure which part of my installation is the source of my problems. As it is working now, I will improve the solution it in the second step. I will post an updated configuration that is more secure, as soon as it is active.

9

Hi tomdx,

I got the solution working with the separated docker network. I did about everything right before, but the local firewall stopped all requests. I have to configure the local firewall to allow access from the docker-container to the host-machine.

Altough I can do it now in both ways, the host-nework approach is less effort becaus I don’t have to configure the firewall if I want to access host-ports from a container. An external access to a container-port is in both cases only possible, if I configure it in the local firewall.

In my opintion, there is no security disadvantage for the host-network approach. The firewall protects both variations equally good.

Could you give me your concerns regarding host-network?

10

Its a best practice thing. If a container exposes a port that only needs to be seen by another container, there is no need to also publish that port on the host machine.

11

Ok, fair point. As I want to have my test-system with an identical setup as my prod-system, I won’t use host-network on both of them.