Setup Wireguard VPN with multiple relay nodes

Staking & Delegation Setup a Stake Pool
1

I have recently set up my stake pool Copenhagen Vertigo following the guides from Coincashew. The recently updated instructions for the 1.26.1 release are excellent – and I will have to buy them a beer.

They also have an easy-to-follow guide on how to setup Wireguard VPN between a BP/local node and a relay/remote node to enhance security. [How to setup WireGuard - CoinCashew]

The wireguard guide is for a single relay node setup and I have added a second relay node to my setup. It is not covered in the guide how to do that, so I have worked a bit on how that could be done. Here are the steps i have found to work to add wireguard in a dual relay node setup. Perhaps it can save time for others to know this.

On the second relay node:
1. Follow the coincashew instructions for installing a relay node
1. Install Wireguard
2. Setup Public / Private Keypair (you will only need the remotenode key pair)
3. Configure Wireguard. A few changes are needed:

  1. change to a different address than the first relay node: e.g. use 10.0.0.3/32 instead of 10.0.0.2/32
  2. use the second relay node’s private key
  3. everything else is the same as for your first relay node (incl. The same local node public key and address)
  4. configure your firewall

'4. Setup autostart with systemd

On the block producer node:
1. Stop wireguard
I tried changing the wg0.conf file while wireguard was running. They were ignored. In the end I ended up copying the content into a new file with the same name while wireguard was stopped, but I think it is enough to stop wireguard while you change the file.
2. Amend the wg0.conf file to look like this:

# local node WireGuard Configuration
[Interface]
# local node address
Address = 10.0.0.1/32
# local node private key
PrivateKey = <local node’s private key>
# local node wireguard listening port
ListenPort = 51820
SaveConfig = true
PostUp   = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o enp9s0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o enp9s0 -j MASQUERADE

# remote node
[Peer]
# remote node's publickey
PublicKey = <relay node 1’s public key>
# remote node's public ip address or dns address
Endpoint = < relay node 1’s public IP address>
# remote node's interface address
AllowedIPs = 10.0.0.2/32
# send a handshake every 21 seconds
PersistentKeepalive = 21

# remote node2
[Peer]
# remote node's publickey
PublicKey = <relay node 2’s public key>
# remote node's public ip address or dns address
Endpoint = < relay node 2’s public IP address>
# remote node's interface address
AllowedIPs = 10.0.0.3/32
# send a handshake every 21 seconds
PersistentKeepalive = 21

'3. restart wireguard

Final step:
Step 4 in the Coincashew guide: Verify that the connection is working and configure the topology.json files with the new addresses.

Comments are welcome. Let me know if I have missed something or there is a better way.

2 Likes
2

BTW… did u updated to 1.26.2 version right?

Cheers,

3

Hi Alex

good to hear from you

No i haven’t updated to 1.26.2. still on 1.26.1. Need to put that on my to do list.

/Keld

4

not to much time left… :slight_smile:

5

Is there a deadline for it? Got my pool up and running this weekend and still have a few monitoring actions.

guess i need to start following and reading release notes :grinning:

6

I think starting with the next epoch it is possible to have issues if u are not using the last version

7

That sounds quite alarming. I will have to look at that.

IOHK’s message is only that it is recommended: “This point release is a recommended upgrade for all stake pool operators.” Release Cardano Node 1.26.2 · input-output-hk/cardano-node (github.com)

Anyway, might as well do it sooner than later. Will try to make it. you wouldn’t happen to know a great guide for creating a backup :slight_smile: ?

8

Ok, turned out to be a walk in the park :crossed_fingers:. Now I have upgraded to 1.26.2.

@Alex: thanks for keeping me posted - now I have added myself to the github watchlist :pray: :muscle:

@ Coincashew and Bebop: thanks for a simple guide. :pray: :muscle:

This community is maturing rapidly!

1 Like
9

Back to the original Thread about Wireguard:
This is how my config looks like (wg0.conf).

[Interface]
ListenPort = 456456
PrivateKey = *************
Address = 10.0.24.10/24

[Peer]
PublicKey = ****************
AllowedIPs = 10.0.24.11

[Peer]
PublicKey = *****************
AllowedIPs = 10.0.24.12

If you are using wg-quick to initialize it takes care about IPtables itself
wg-quick up wg0

10

Glad to see you got wireguard working. I ended giving up after a while. Didn’t seem to work when I followed the coincashew guide. Curious if you changed something in the guide for the single relay to get it working?

11

Also, for grafana and Prometheus install did you have to change anything to get it to work with wireguard?

12

I did not follow the Coincashew guide. Using Wireguard for long time and just did it like always.

13

Wireguard is acting like a normal network interface.
So you just need to allow the according ports on the Firewall for this network interface and use the VPN IP instead of the public one for your connections. Instead nothing needs to be changed.

14

Hey doc

I don’t recall changing anything. basically copying and filling in the right addresses. this was during the weekend (~april 17). If you managed to implement following the overall guide, then I don’t see why it shouldn’t work.

Only thing I remember where I had to concentrate a bit was around the keys. if you run step 2 in the guide on both the local and remote node, then you get 4 sets of keys - 2 on each node. But you only need 1 set per nodes, of course, and if you don’t concentrate then you might end up using the wrong keys in step 3. And if you discover a mistake and change the config afterwards, you have stop wireguard otherwise I found that it ignores changes to the config file. Somehow i am drawn to such trivial mistakes like a magnet ;-D

15

Thanks, I might retry it again this weekend on my testnet pool. Likely I made a mistake with the keys. Thanks for the help!

16

What is the scope of configuring a vpn as long the relay is connected to the public network?

17

Hi guys, got the wireguard vpn working on my testnet between the producer and relay which seems to be working. Problem I’ve run into is that prometheus doesnt seem to be scraping my producer node as I’m not getting any metrics on my grafana dash from it. Wondering if I did something wrong?

My BP is set up as the wireguard local node and my first relay as the wireguard peer. The BP node is an exporter and the relay my grafana server.

I’m thinking I did something wrong in the prometheus config on the grafana server. On coincashew is made up as follows:

cat > prometheus.yml << EOF
global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    monitor: 'codelab-monitor'

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label job=<job_name> to any timeseries scraped from this config.
  - job_name: 'prometheus'

    static_configs:
      - targets: ['localhost:9100']
      - targets: ['<block producer public ip address>:9100']
      - targets: ['<block producer public ip address>:12798']
        labels:
          alias: 'block-producer-node'
          type:  'cardano-node'
      - targets: ['localhost:12798']
        labels:
          alias: 'relaynode1'
          type:  'cardano-node'
EOF
sudo mv prometheus.yml /etc/prometheus/prometheus.yml

I changed the <block producer public ip address> to my bp vpn (10.0.0.1) but still doesn’t seem to work.

Any ideas what to do?

(Was wondering what Prometheus exporter to use on my BP? I see there is a wireguard version online?)

18

Hi doc

looks alright.

The relay node and the BP are connecting fine via port 6000 using wireguard? and you see relay node data in your grafana dashboard?

19

when I remove port 6000 from the BP I end up dropping my “IN” connection from my relay node. I have the topology files on both relay and BP pointing to the wireguard VPN. Not sure what I’m doing wrong.

Topology files:

Relay:
“addr”: “10.0.0.1”, #vpn address for BP
“port”: 6000,
“valency”: 1

BP:
“addr”: “10.0.0.2”, #VPN address for relay
“port”: 6000,
“valency”: 1

I get the impression the IN to the BP is still trying to go through the regular port. Unsure how to force it through the VPN

Thanks again for any help!

20

Hi doc

I am running out of ideas - understand if you are frustrated ;-). you have opened port 51820/udp and can ping 10.0.0.1 from the relaynode?

Here are my settings and status. don’t know if it brings any insights:

The BP node (my 10.0.0.1):

sudo ufw status numbered:

To                         Action      From
 --                         ------      ----

[ 1] 22/tcp ALLOW IN Anywhere
[ 2] 51820/udp ALLOW IN Anywhere

nano wg0.conf:

[as above - the initial post]

sudo wg:

interface: wg0
public key:
private key: (hidden)
listening port: 51820

peer:
endpoint: :51820
allowed ips: 10.0.0.3/32
latest handshake: 30 seconds ago
transfer: 220.53 MiB received, 145.18 MiB sent
persistent keepalive: every 21 seconds

peer:
endpoint: <IPv4 address of relay - the grafana node>:51820
allowed ips: 10.0.0.2/32
latest handshake: 31 seconds ago
transfer: 100.79 MiB received, 96.58 MiB sent
persistent keepalive: every 21 seconds

nano mainnet-topology.json

{
“Producers”: [
{
“addr”: “10.0.0.2”,
“port”: 6000,
“valency”: 2
},
{
“addr”: “10.0.0.3”,
“port”: 6000,
“valency”: 2
}
]
}

On the relay node running grafana (my 10.0.0.2):

nano prometheus.yml:

global:
scrape_interval: 15s # By default, scrape targets every 15 seconds.

Attach these labels to any time series or alerts when communicating with

external systems (federation, remote storage, Alertmanager).

external_labels:
monitor: ‘codelab-monitor’

A scrape configuration containing exactly one endpoint to scrape:

Here it’s Prometheus itself.

scrape_configs:

The job name is added as a label job=<job_name> to any timeseries scraped from th> - job_name: ‘prometheus’

static_configs:
  - targets: ['localhost:9100']
  - targets: ['10.0.0.1:9100']
  - targets: ['10.0.0.1:12798']
    labels:
      alias: 'Vertigo block-producer'
      type:  'cardano-node'
  - targets: ['localhost:12798']
    labels:
      alias: 'Vertigo relay1'
      type:  'cardano-node'
  - targets: ['<ip4 address of my second relay>:9100']
  - targets: ['<ip4 address of my second relay>:12798']
    labels:
      alias: 'Vertigo relay2'
      type:  'cardano-node'

sudo wg

interface: wg0
public key:
private key: (hidden)
listening port: 51820

peer:
endpoint: :51820
allowed ips: 10.0.0.1/32
latest handshake: 3 seconds ago
transfer: 96.25 MiB received, 101.13 MiB sent
persistent keepalive: every 21 seconds

nano mainnet-topology.json

{ “resultcode”: “201”, “networkMagic”: “764824073”, “ipType”:4, “requestedIpVersion”:> { “addr”: “10.0.0.1”, “port”: 6000, “valency”: 1 } ,
<+external relay nodes>

on the relay node i have more port opened, including the 51820/udp

1 Like