Why do nodes have a second connection to each other?

Staking & Delegation Operate a Stake Pool
,
1

Under peer analysis in gLiveView, why do my nodes have two connections to each other?

The correct ports for each of the nodes are 64100, 64101 & 64102. The second connection is using a seemingly random port.

Here’s a couple screenshots to illustrate:

Screenshot

Kind regards
Adam

2

I noticed this behavior also… I didn’t tried to restart the node… did u?

3

Yes, it persists after a restart.

Strange too because the random ports should be blocked by firewall.

4

I belive there are the outgoing ports not incoming ports

5

I do not use gLiveView, but yes as @Alexd1985 wrote each BP will have one outgoing connection to relay, that what you put in topology file of BP and one incoming connection from relay, because again you put in topology file of relay BP there. Outgoing connections are from ephemeral ports or high random port to port what you configure.

Usually you allow all outgoing traffic, that why it is not blocked by FW, but technically you can block outgoing traffic too.

Here short manual:

http://www.tcpipguide.com/free/t_TCPIPClientEphemeralPortsandClientServerApplicatio-3.htm

Additionally your FW has following rule:

iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

This means that we do allow connections what were established, basically if we opened connection from BP to relay port 64102, then we will accept replies from any port from relay. That why your BP allows connection at port 40431 for example.

1 Like
6

Thanyou Jurijs for your helpful explanation!

1 Like
7

Cardano nodes currently use half-duplex connections. These half-duplex connections can only “pull” blocks.

Therefore your block producer will need:

  • pull connections to your relays in order to pull the latest blocks from your relays that they have in turn received from the rest of cardano network (set this up in your block producer topology file)
  • incoming pull connections from your relays in order for your relays to pull the blocks your block producer produces (set this up in your relay topology files)

Your relays will need:

  • pull connections to your block producer (set this up in your relay topology file)
  • pull connections to other relays on the cardano network (set this up in your relay topology file)
  • incoming pull connections from other cardano relays into your relays (this is set up by other relays in their toplogy files).
    In order to have this last thing happen you need to ping api.clio.one with your relay topology data otherwise no other cardano relays will know how to contact your relay and they won’t set up any pull connections to your relay.

This all changes with P2P mode roll-out (hopefully soon) because then nodes can set up full-duplex connections and they can automatically discover each other.

2 Likes
8

Hello, well it’s a standard behaviour in TCP/IP networking. Every out connection has a pseudo-random source port. It’s not something about cardano-node at all

9

Yep, high random source port is totally usual.

Needing two connections – one for each direction – not so much.

10

Hey 7.4d4, thanks for this great explanation of how the nodes work. In order to see the whole picture, I’d like to know the purpose of registering the relays on-chain with the stake pool. In other words, what would happen if the registered relays are different than the ones actually having the BP in their topology file, counting that they are already connected to other relays?

Cheers!

11

Registering the relays on chain enables other relays to know how to connect to your relays.

Current mainnet:

The current setup, used by most people on mainnet, is somewhat circumventing the need for this on-chain registration through the use of the “topology update service”: https://api.clio.one/htopology/v1/

Essentially the way this topology update service works is as follows:

  • Every hour you have your relay node send api.clio.one its current block no, port no and host name (eg. via a cron job). Each time you send this info api.clio.one gets the IP address too. After a few hours to ensure your relay doesn’t disappear, api.clio.one adds the (IP and port no) to its database of known relays. api.clio.one also adds the chain registered relays to its database.
  • Once your relay is known to api.clio.one you can then request a list of relays from api.clio.one and it will send you the data in the form of a topology.json file. To generate this list api.clio.one pulls data from its database of known relays and collates the information according to geographical location (calculated from IP addresses) and sends you a diverse mix of relays. It chooses the relays for you to preference closer geolocated relays but also to give you some on the other side of the world. From what I can see, api.clio.one seems to do this very well and fairly.

In other words, this means that even if you didn’t register a particular relay IP address and port combination on chain, you can still get your relay known to others using this api.clio.one service.

P2P mode:

This will change when P2P mode gets used more. When you configure your node to use P2P mode it can seek relays to connect with from the chain registered data itself. Furthermore, it seems that P2P relays can tell each other about other relays that they know. The P2P mode relays then propagate information about each other and monitor which relays send them new block data quickest so that they can preferentially connect with these relays. The P2P protocol design and actually the entire node to node communication networking stack seems to be brilliantly designed (but I am no expert).

Also P2P mode relays can do full duplex connections instead of the half duplex. So P2P mode relays can send transaction and block data both ways.

Once P2P mode is switched on fully, the api.clio.one topology service will no longer be needed. The chain registered data will be a starting point but even that it seems may not be essential. Still, I would definitely recommend that you register your relays on chain because you need other relays connecting to yours so your pool’s blocks propagate quickly.

I have been running several nodes in P2P mode on mainnet flawlessly for months. I still ping api.clio.one every hour from these P2P nodes so other relays (not in P2P mode) can know about the IP address + Port combinations. But, I don’t need to use api.clio.one to configure the mainnet-topology.json files for these P2P mode relays.

1 Like