Failed to start all required subscriptions

Staking & Delegation Setup a Stake Pool
1

I’m running two docker containers (one for block producer, one for relay), both are connected to the same docker network (bridge), both have their ports opened, but starting the cardano-block It’s unable to connect to the relay.

Any ideas?

Sample error is:

[95b6abb9:cardano.node.IpSubscription:Info:54] [2021-05-20 00:44:17.42 UTC] IPs: 0.0.0.0:0 [172.17.0.2:3001] Connection Attempt Start, destination 172.17.0.2:3001
[95b6abb9:cardano.node.IpSubscription:Error:54] [2021-05-20 00:44:17.42 UTC] IPs: 0.0.0.0:0 [172.17.0.2:3001] Connection Attempt Exception, destination 172.17.0.2:3001 exception: Network.Socket.connect: <socket: 25>: does not exist (Connection refused)
[95b6abb9:cardano.node.IpSubscription:Info:54] [2021-05-20 00:44:17.42 UTC] IPs: 0.0.0.0:0 [172.17.0.2:3001] Closed socket to 172.17.0.2:3001
[95b6abb9:cardano.node.ErrorPolicy:Notice:51] [2021-05-20 00:44:17.42 UTC] IP 172.17.0.2:3001 ErrorPolicySuspendConsumer (Just (ConnectionExceptionTrace Network.Socket.connect: <socket: 25>: does not exist (Connection refused))) 20s
[95b6abb9:cardano.node.IpSubscription:Error:53] [2021-05-20 00:44:17.44 UTC] IPs: 0.0.0.0:0 [172.17.0.2:3001] Failed to start all required subscriptions

Docker configuration:

  • $ docker ps
CONTAINER ID   IMAGE                         COMMAND       CREATED        STATUS        PORTS                                       NAMES
0af457255172   ubuntu                        "/bin/bash"   19 hours ago   Up 19 hours                                               cold-machine
95b6abb97ac1   cardano-block-v2:production   "/bin/bash"   47 hours ago   Up 42 hours   0.0.0.0:6000->6000/tcp, :::6000->6000/tcp   cardano-block
a643ac348cfd   ubuntu                        "bash"        2 days ago     Up 42 hours   0.0.0.0:3001->3001/tcp, :::3001->3001/tcp   cardano-relay
  • $ docker inspect bridge
"Containers": {
    "95b6abb97ac1c619443870781b1e9fa17c8a4d30bc991371331a11cb7c9c0583": {
        "Name": "cardano-block",
        "EndpointID": "ef09463393d166f46c4e108176488f726fa4f0d6448eda3452ca0bb52fff792b",
        "MacAddress": "02:42:ac:11:00:03",
        "IPv4Address": "172.17.0.3/16",
        "IPv6Address": ""
    },
    "a643ac348cfda479c31f979ade79c5a6e0e0fd41e2b904c24bb7ab3ea463e61d": {
        "Name": "cardano-relay",
        "EndpointID": "2fa275f68e743ab907cc94d0d2da43024b13da550930768d11a760abc5dd371e",
        "MacAddress": "02:42:ac:11:00:02",
        "IPv4Address": "172.17.0.2/16",
        "IPv6Address": ""
    }
  • mainnet-topology.json (block-producer)
{
  "Producers": [
    {
      "addr": "172.17.0.2",
      "port": 3001,
      "valency": 1
    }
  ]
}
  • mainnet-topology (relay node)
{
  "Producers": [
    {
      "addr": "relays-new.cardano-mainnet.iohk.io",
      "port": 3001,
      "valency": 2
    },
    {
      "addr": "172.17.0.3",
      "port": 6000,
      "valency": 1
    }
  ]
}

I believe the relay node is up and running and working, I guess the cardano-block container just can’t reach to it for some reason?

$ ./startRelayNode1.sh

Listening on http://127.0.0.1:12798
[a643ac34:cardano.node.networkMagic:Notice:5] [2021-05-20 00:44:02.52 UTC] NetworkMagic 764824073
[a643ac34:cardano.node.basicInfo.protocol:Notice:5] [2021-05-20 00:44:02.52 UTC] Byron; Shelley
[a643ac34:cardano.node.basicInfo.version:Notice:5] [2021-05-20 00:44:02.52 UTC] 1.27.0
[a643ac34:cardano.node.basicInfo.commit:Notice:5] [2021-05-20 00:44:02.52 UTC] 8fe46140a52810b6ca456be01d652ca08fe730bf
[a643ac34:cardano.node.basicInfo.nodeStartTime:Notice:5] [2021-05-20 00:44:02.52 UTC] 2021-05-20 00:44:02.523784248 UTC
[a643ac34:cardano.node.basicInfo.systemStartTime:Notice:5] [2021-05-20 00:44:02.52 UTC] 2017-09-23 21:44:51 UTC
[a643ac34:cardano.node.basicInfo.slotLengthByron:Notice:5] [2021-05-20 00:44:02.52 UTC] 20s
[a643ac34:cardano.node.basicInfo.epochLengthByron:Notice:5] [2021-05-20 00:44:02.52 UTC] 21600
[a643ac34:cardano.node.basicInfo.slotLengthShelley:Notice:5] [2021-05-20 00:44:02.52 UTC] 1s
[a643ac34:cardano.node.basicInfo.epochLengthShelley:Notice:5] [2021-05-20 00:44:02.52 UTC] 432000
[a643ac34:cardano.node.basicInfo.slotsPerKESPeriodShelley:Notice:5] [2021-05-20 00:44:02.52 UTC] 129600
[a643ac34:cardano.node.basicInfo.slotLengthAllegra:Notice:5] [2021-05-20 00:44:02.52 UTC] 1s
[a643ac34:cardano.node.basicInfo.epochLengthAllegra:Notice:5] [2021-05-20 00:44:02.52 UTC] 432000
[a643ac34:cardano.node.basicInfo.slotsPerKESPeriodAllegra:Notice:5] [2021-05-20 00:44:02.52 UTC] 129600
[a643ac34:cardano.node.basicInfo.slotLengthMary:Notice:5] [2021-05-20 00:44:02.52 UTC] 1s
[a643ac34:cardano.node.basicInfo.epochLengthMary:Notice:5] [2021-05-20 00:44:02.52 UTC] 432000
[a643ac34:cardano.node.basicInfo.slotsPerKESPeriodMary:Notice:5] [2021-05-20 00:44:02.52 UTC] 129600
[a643ac34:cardano.node.addresses:Notice:5] [2021-05-20 00:44:02.52 UTC] [SocketInfo 0.0.0.0:6000]
[a643ac34:cardano.node.diffusion-mode:Notice:5] [2021-05-20 00:44:02.52 UTC] InitiatorAndResponderDiffusionMode
[a643ac34:cardano.node.dns-producers:Notice:5] [2021-05-20 00:44:02.52 UTC] [DnsSubscriptionTarget {dstDomain = "relays-new.cardano-mainnet.iohk.io", dstPort = 3001, dstValency = 2}]
[a643ac34:cardano.node.ip-producers:Notice:5] [2021-05-20 00:44:02.52 UTC] IPSubscriptionTarget {ispIps = [172.17.0.3:6000], ispValency = 1}

(Currently following tutorial from CoinCashew)

1 Like
2

You can just open up all your ports since you have 0.0.0.0:6000. Try 0.0.0.0/16:0-65535

[95b6abb9:cardano.node.ErrorPolicy:Notice:51] [2021-05-20 00:44:17.42 UTC] IP 172.17.0.2:3001 ErrorPolicySuspendConsumer (Just (ConnectionExceptionTrace Network.Socket.connect: <socket: 25>: does not exist (Connection refused))) 20s
This is an firewall issue more than likely.

3

I would use the cold container to netcat or telnet and would test the port openings netcat -vzn 172.17.0.2 3001 ro telnet 172.17.0.2 3001

4

I’ve tried adding the ‘ufw’ fire wall rules:

3001                       ALLOW       Anywhere                  
172.17.0.3 6000            ALLOW       172.17.0.2

Even tried disabling the firewall all together, but still getting

netcat: connect to 172.17.0.2 port 3001 (tcp) failed: Connection refused

5

as a workaround try to start the containers on host network…

6

Actually running

$ netcat -vzn 0.0.0.0 3001

I get

Connection to 0.0.0.0 3001 port [tcp/*] succeeded!

Which is so weird, I’ve tried my host ip and the container ip, both connections get refused, but ip 0.0.0.0 works? I’m confused

7

btw, how you specified the --host-addr for cardano run

8

I think this is mainly a docker issue. I’m able to connect to the container from the host, but not from outside the host, which is the issue because then the relay node is not accessible from outside the host. I’m struggling to find forums that explain how to connect to a container from outside the host, but most online solutions are for Virtual Machines running the docker, which is not my case. I’m running the containers from a physical server running Ubuntu

9

Actually I’m not too sure if it’s just a docker issue. My cardano-node run script looks like this:

#!/bin/bash
DIRECTORY=/root/cardano-my-node
PORT=3001
HOSTADDR=0.0.0.0
TOPOLOGY=${DIRECTORY}/mainnet-topology.json
DB_PATH=${DIRECTORY}/db
SOCKET_PATH=${DIRECTORY}/db/socket
CONFIG=${DIRECTORY}/mainnet-config.json
/usr/local/bin/cardano-node run --topology ${TOPOLOGY} --database-path ${DB_PATH} --socket-path ${SOCKET_PATH} --host-addr ${HOSTADDR} --port ${PORT} --config ${CONFIG}

I’ve tried changing the HOSTADDR to 172.17.0.2 but still can’t connect to this container from outside host… :cry:

10

The launch config looks correct… Maybe you can try without docker? Is there a local firewall on the os ?

11

I only have one physical server so I can’t really have a relay node and a block producer node on the same machine so I have to use docker.

Also no, I turned off ufw firewall so that shouldn’t be causing the problems.

12

You can have two different nodes running on the same system. in my case I have a node running on mainnet and other is running on testnet… without docker…

13

Will have to agree with @laplasz . You can have it running on the same machine but with different ports. Just risky as your relay dns/ip will be available to the public and could bring your network down in an event of ddos.

14

Where would you suggest to host the relay node? I am trying to set it up at home, but I don’t know where else to host it. A cheap cloud solution like Linode server with 8 gb would cost about $40 a month :confused:

15

@Alexd1985 may have some options.
I have 1 relay on AWS, 1 on Azure, and 1 at home on an older laptop. The AWS/Azure are costly indeed.

16

@KingTChoka u can try vpn M SSD plan

17

Thanks for the suggestions, I guess I’ll look towards cloud solutions perhaps. But don’t home ISP’s block DDOS by default anyways?

18

U need to search for the best solution for u

19

Perhaps some do, but to my knowledge by default no. If you have another computer laying around, maybe you can use it or install an some virtual machines on your current one.

Everyone situation is different as Alex mentioned.

20

@Alexd1985
I’m having the same issue after changing the config files i’m getting this error
May 25 14:18:20 ip- cardano-node[112208]: [ip-172-3:cardano.node.ErrorPolicy:Warning:57] [2021-05-25 14:18:20.08 UTC] IP 18.158.196.38:3001 ErrorPolicySuspendPeer (Just (ApplicationExceptionTrace (HeaderError (At (Block {blockPointSlot = SlotNo 4492800, blockPointHash = aa83acbf5904c0edfe4d79b3689d3d00fcfc553cf360fd2229b98d464c28e9de})) (HeaderProtocolError (HardForkValidationErrFromEra S (Z (WrapValidationErr {unwrapValidationErr = ChainTransitionError [OverlayFailure (VRFKeyBadNonce (Nonce “81e47a19e6b29b0a65b9591762ce5143ed30d0261e5d24a3201752506b20f15c”) (SlotNo 4492800) (Nonce “40710bcc4e024224641dc12fe1e5e478d2c3d3aa7ad4f032283ddcc73b2ddb03”) (CertifiedVRF {certifiedOutput = OutputVRF {getOutputVRFBytes = “6\236Sx\209\245\EOT\SUBY\235\141\150\230\GS\233o\tP\251A\180\159\245\DC1\247\188\DEL\209\t\212&8>\GS$\190p4\230t\156f\DC2p\r\213\206\176\198ew\184\138\EM\174(k\DC3!\209[\206\SUB\183&6”}, certifiedProof = CertPraosVRF “@Z\163p\255\NUL\149D\162\190J\165\188R\196Ec3\244\249\182W\GSf\197Y\r.V)\208\139:6\t\244\189\ENQ\STX\237]\v\225\171\219\DEL*\183j\174\174G\254\DC1\ESC\ETX5\164\228\222\246F\147\SYN’\148\184\211\193\202qP\SI\SYN\177\226DrL\ETX”}))]})))) (Tip (SlotNo 4492780) bfdb854da7be13d121ad66f3ebdd50facc021d1d691835a7b41025813d429626 (BlockNo 4490491)) (Tip (SlotNo 30385990) 394e8e4d8cefd896390212d5d80085d225803c5c8cfd433a62d2d12ee8d3e3ca (BlockNo 5764503))))) 200s 200s

Can you help me ?