Setup Wireguard VPN with multiple relay nodes

Hi keldstehr,

Thanks for the input, looks exactly the same as I have. I might give it a rest a bit and try again in a few days. Really appreciate the help!

Hi, I would like to bring this topic back up because I believe I am facing the same issues.

I followed the Coincashew Wireguard instruction as well as this thread.

My Wireguard seems to be set up properly, when I “sudo wg” I see data sent and received, I can ping on 10.0.0.1 successfully, and vice versa from the relay nodes.

I have updated my BP Topology and Relay topology:
image

My BP node will connect to incoming peer but cannot reach the relay nodes:

image

ALSO: in order to get this far I had to change my BP node’s address in startBlockProducingNode.sh, without this there was zero communication:
HOSTADDR=10.0.0.1

Now if I make this same change to the startup script in the relay nodes they will connect with the BP but no longer receive from external nodes.

-Is it ok that all of my nodes are using port 1600 as opposed to the normal 6000? I also tried 6000 but didn’t seem to help much.
-Should I be updating my node startup scripts with the 10.0.0.x addresses? I don’t see this mentioned anywhere but it seemed to have helped my situation.

Any suggestions would be welcomed.

Hi Benagain,

its funny I kind of forgot about wireguard and then last night I started playing around with it again.

I was able to get my nodes on my testnet talking to each other with wireguard and my BP node is getting in/out connections now and it seems to be running transactions correctly.

Problem I have now is that for some reason I cant seem to scrape metrics over the VPN from prometheus. I checked my prometheus page on my node and I get the following error:

Get http://10.0.1.2:12798/metrics: context deadline exceeded

tried increasing the scrape interval but no success
has anyone else run into this problem?

Hi keldstehr,

Started tinkering with wireguard again to see if I could get it working. Able to get a stable connection now between my testnet BP and relay but noticed I wasn’t able to scrape metrics from Prometheus over the VPN. Last night I port forwarded the EKG and Prometheus ports from my BP and relay to the VPN address of my offsite grafana server and suddenly my metrics (most of them, minus mempool and transactions) suddenly appeared!

Did you have to forward your ports through the VPN for grafana/Prometheus to work? I thought that the wg-quick command did all that automatically by I think I was mistaken

Hey doc_keiger,

Did you make any changes to the HOSTADDRESS in your node startup scrips? I don’t see it mentioned in the script but for some reason my node will not communicate without changing it to 10.0.0.1 on my BP node. Maybe my iptables are not updating the the wg0.conf setting?

I dont think I did. I did have to change the topology files in each in order to get it working.

For the BP you just go into your mainnet-topology.json and change the IP to the wireguard address of your relay:

nano $NODE_HOME/mainnet-topology.json
 {
    "Producers": [
      {
        "addr": "<WIREGUARD RELAYNODE IP ADDRESS>",
        "port": 6000,
        "valency": 1
      }
    ]
  }

Then restart your BP node:

sudo systemctl restart cardano-node

On the relay, you need to change the relay-topology_pull.sh file. Specifically, you need to change the Block producing IP to your VPN Block Prod IP:

nano $NODE_HOME/relay-topology_pull.sh

 
#!/bin/bash

BLOCKPRODUCING_IP=<WIREGUARD BLOCK PRODUCERS IP ADDRESS>

BLOCKPRODUCING_PORT=6000

curl -s -o $NODE_HOME/${NODE_CONFIG}-topology.json "https://api.clio.one/htopology/v1/fetch/?max=20&customPeers=\${BLOCKPRODUCING_IP}:\${BLOCKPRODUCING_PORT}:1|relays-new.cardano-mainnet.iohk.io:3001:2"

Then execute the script and restart the node

./relay-topology_pull.sh
sudo systemctl restart cardano-node

This should make your nodes talk to each other

1 Like

Let me know if you have any issues or if you have any success with getting your grafana metrics to work properly over wireguard

Thanks Doc,

Your Post was excellent. I definitely did not have my my relay-topology_pull.sh correct. Alas, it did not fix my communication issue though.

As far as I can tell I have everything configured exaclty as specified in the original post on this thread and your examples. I can ping between 10.0.0.1 - 10.0.0.2 and 10.0.0.3. My nodes will even communicate with each other over wireguard… if I change the startRelayNode1.sh host address to 10.0.0.x. That is not the correct behavior but may be the best clue to root causing this. So I guess how does the node recieve connections from both my BP on 10.0.0.1 and the external nodes simultaneously (iptable?), because right now I have an either (BP node) or (external nodes) state via my startRelayNode1.sh.

Hmm, that’s odd. Going to play around with it a bit more tonight and tomorrow. I’ll let you know if I find something else to adjust.

Just want to make sure but you did also remember to open your wireguard ports on the respective nodes. (I know I forgot to do it initially and kicked myself later when I realized I missed it).