Relay Node - Topology Updater Always Returns "IP is not (yet) allowed to fetch this list"

Staking & Delegation Setup a Stake Pool
1

Hi everyone

I have been playing with setting up a stake pool on the testnet and no matter what I do the topology updater script does not work for my configuration. I am aware that after four hours I am supposed to see the “glad you’re staying with us” message, but I never see this, it continues with “welcome to the topology” and my topology file always contains the message: “IP is not (yet) allowed to fetch this list”

I am running this on my home network with a configuration that looks like this:

Untitled Diagram-2

Additional Information:

  • Stake Pool name: JB2
  • Blockchain is synced on both nodes
  • All nodes have firewalls currently disabled
  • Relay configuration is present in the http://explorer.cardano-testnet.iohkdev.io/relays/topology.json file.
  • Transactions are being processed by both the relay and core nodes.
  • Router logs are showing successful port forwarding requests to the relay node.

I have been running this for a couple of weeks without success, so any help the community can give me on this issue would be much appreciated.

Thanks in advance

1 Like
2

Hi Mick,

I assume the GENESIS_JSON variable is correctly configured in topologyUpdater.sh?

3

Also do you have the script configured to run once every hour?

4

The GENESIS_JSON file is referenced in the env file:

GENESIS_JSON="${CNODE_HOME}/files/genesis.json"

This file looks to be correct for testnet:

cat ${CNODE_HOME}/files/genesis.json
{
  "activeSlotsCoeff": 0.05,
  "protocolParams": {
    "protocolVersion": {
      "minor": 0,
      "major": 2
    },
    "decentralisationParam": 1,
    "eMax": 18,
    "extraEntropy": {
      "tag": "NeutralNonce"
    },
    "maxTxSize": 16384,
    "maxBlockBodySize": 65536,
    "maxBlockHeaderSize": 1100,
    "minFeeA": 44,
    "minFeeB": 155381,
    "minUTxOValue": 1000000,
    "poolDeposit": 500000000,
    "minPoolCost": 340000000,
    "keyDeposit": 2000000,
    "nOpt": 150,
    "rho": 0.003,
    "tau": 0.20,
    "a0": 0.3
  },
  "genDelegs": {
    "2f56e87d67b8e5216582cfeb95dbdc9083110a3ef68faaa51bef3a80": {
      "delegate": "bd5933d3c5417f17a64c7214711a26abc3bc03e2c90dc1bb38e0c39f",
      "vrf": "9a0b0f537874d089cedfa9e250150405e47ea29acee87c40a223ae0a175d26f8"
    },
    "514e81afb082fce01678809eebd90eda4f7918354ec7d0433ad16274": {
      "delegate": "eff1b5b26e65b791d6f236c7c0264012bd1696759d22bdb4dd0f6f56",
      "vrf": "e6f70fb10c7523aa76648e20d17e65fd9b2ed53960fbd20b308f223b703f2e23"
    },
    "2fca486b4d8f1a0432f5bf18ef473ee4294c795a1a32e3132bc6b90f": {
      "delegate": "de665a71064706f946030505eae950583f08c316f0f58997961092b1",
      "vrf": "c3fde629add60e30142cd7ef3c680610975208b08aee42203a5c40ad5992e8f6"
    },
    "4ee98623920698b77c1c7f77288cbdac5f9011ff8970b1f507567d0d": {
      "delegate": "bf07107c6f632de95e34af7e009d2aafa19916c7ba89b944fbedcd72",
      "vrf": "9d7d12e3d6b02835be3e76cfc6ae93d937035ee0e006d04a0eef9dea19754e21"
    },
    "0d06d2547ed371fdf95fb5c4c735eecdd53e6a5bb831561bd0fcfd3d": {
      "delegate": "6df3e1b4b8a84c63c805076a85e5aa00924997a4eae85fddf0aee3ca",
      "vrf": "0774e5810fe02a014ec97ef424797172f2b8c5dcfb6e4cfc98b411c31d5096d8"
    },
    "581e23030b6038bae716e5d64b9e053db10541b12e6b0b4eff485454": {
      "delegate": "b0dca078b823cde627da136200d6618c49ad712b77972a1c5e135763",
      "vrf": "16a4e883b72ddbd09a4f8a1170fc346ab11e4202f814faa73e9d2433ee03e7b0"
    },
    "e5f27655371b54aed91cc916b2569060978be80056768fee2cc5ce1b": {
      "delegate": "b3873a254459f506e47b9a252ee7912e538b364447f31576a170db65",
      "vrf": "cc5c897fdf5db0017326656fe35aeb20c72b175540793f9b9b8dc9ade001bbc4"
    }
  },
  "updateQuorum": 5,
  "networkId": "Testnet",
  "initialFunds": {},
  "maxLovelaceSupply": 45000000000000000,
  "networkMagic": 1097911063,
  "epochLength": 432000,
  "systemStart": "2019-07-24T20:20:16Z",
  "slotsPerKESPeriod": 129600,
  "slotLength": 1,
  "maxKESEvolutions": 62,
  "securityParam": 2160
}
5

I have configured using systemd, so have the hourly timer set:

NEXT                        LEFT        LAST                        PASSED       UNIT                         ACTIVATES                     
Thu 2021-08-26 09:01:00 UTC 13min left  Thu 2021-08-26 08:00:57 UTC 46min ago    cnode-tu-push.timer          cnode-tu-push.service
6

When running curl https://ipinfo.io/ip from the relay node, does it resolve to the same IP address the dynamic dns is pointing to?

7

Yes that resolves as the correct IP address.

8

hmm maybe you can copy/paste the topologyUpdater script, just to be sure it’s not an outdated version.

User @werkof is the creator of topologyUpdater. In case he has time, maybe he can have a peak at the logs.

9

Sure thing, here it is:

cat topologyUpdater.sh
#!/usr/bin/env bash
# shellcheck disable=SC2086,SC2034
# shellcheck source=/dev/null

PARENT="$(dirname $0)"
[[ -f "${PARENT}"/env ]] && . "${PARENT}"/env offline

######################################
# User Variables - Change as desired #
######################################

CNODE_HOSTNAME="dynamic-dns-hostname"  # (Optional) Must resolve to the IP you are requesting from
CNODE_VALENCY=1             # (Optional) for multi-IP hostnames
MAX_PEERS=15                # Maximum number of peers to return on successful fetch (note that a single peer may include valency of up to 3)
CUSTOM_PEERS="10.1.1.20,3001|10.1.1.21,3001"        # *Additional* custom peers to (IP,port[,valency]) to add to your target topology.json
                            # eg: "10.0.0.1,3001|10.0.0.2,3002|relays.mydomain.com,3003,3"
#BATCH_AUTO_UPDATE=N        # Set to Y to automatically update the script if a new version is available without user interaction

######################################
# Do NOT modify code below           #
######################################

PARENT="$(dirname $0)"
[[ -f "${PARENT}"/.env_branch ]] && BRANCH="$(cat ${PARENT}/.env_branch)" || BRANCH="master"

usage() {
  cat <<-EOF
		Usage: $(basename "$0") [-b <branch name>] [-f] [-p]
		Topology Updater - Build topology with community pools

		-f    Disable fetch of a fresh topology file
		-p    Disable node alive push to Topology Updater API
		-b    Use alternate branch to check for updates - only for testing/development (Default: master)
		
		EOF
  exit 1
}

TU_FETCH='Y'
TU_PUSH='Y'

while getopts :fpb: opt; do
  case ${opt} in
    f ) TU_FETCH='N' ;;
    p ) TU_PUSH='N' ;;
    b ) echo "${OPTARG}" > "${PARENT}"/.env_branch ;;
    \? ) usage ;;
  esac
done
shift $((OPTIND -1))

[[ -z "${BATCH_AUTO_UPDATE}" ]] && BATCH_AUTO_UPDATE=N

#######################################################
# Version Check                                       #
#######################################################
clear

if [[ ! -f "${PARENT}"/env ]]; then
  echo -e "\nCommon env file missing: ${PARENT}/env"
  echo -e "This is a mandatory prerequisite, please install with prereqs.sh or manually download from GitHub\n"
  exit 1
fi

. "${PARENT}"/env offline &>/dev/null # ignore any errors, re-sourced later

if [[ "${UPDATE_CHECK}" == "Y" ]] && [[ "${BATCH_AUTO_UPDATE}" == "N" ]]; then
  echo "Checking for script updates..."
  # Check availability of checkUpdate function
  if [[ ! $(command -v checkUpdate) ]]; then
    echo -e "\nCould not find checkUpdate function in env, make sure you're using official guild docos for installation!"
    exit 1
  fi
  # check for env update
  ! checkUpdate env ${BATCH_AUTO_UPDATE} && exit 1
  ! checkUpdate topologyUpdater.sh ${BATCH_AUTO_UPDATE} && exit 1
  # source common env variables in case it was updated
  . "${PARENT}"/env offline &>/dev/null
  case $? in
    0) : ;; # ok
    2) echo "continuing with topology update..." ;;
    *) exit 1 ;;
  esac
fi

# Check if old style CUSTOM_PEERS with colon separator is used, if so convert to use commas
if [[ -n ${CUSTOM_PEERS} && ${CUSTOM_PEERS} != *","* ]]; then
  CUSTOM_PEERS=${CUSTOM_PEERS//[:]/,}
fi

if [[ ${TU_PUSH} = "Y" ]]; then
  fail_cnt=0
  while ! blockNo=$(curl -s -f -m ${EKG_TIMEOUT} -H 'Accept: application/json' "http://${EKG_HOST}:${EKG_PORT}/" 2>/dev/null | jq -er '.cardano.node.metrics.blockNum.int.val //0' ); do
    ((fail_cnt++))
    [[ ${fail_cnt} -eq 5 ]] && echo "5 consecutive EKG queries failed, aborting!"
    echo "(${fail_cnt}/5) Failed to grab blockNum from node EKG metrics, sleeping for 30s before retrying... (ctrl-c to exit)"
    sleep 30
  done
fi

if [[ -n ${CNODE_HOSTNAME} && "${CNODE_HOSTNAME}" != "CHANGE ME" ]]; then
  T_HOSTNAME="&hostname=${CNODE_HOSTNAME}"
else
  T_HOSTNAME=''
fi

if [[ ${TU_PUSH} = "Y" ]]; then
  if [[ ${IP_VERSION} = "4" || ${IP_VERSION} = "mix" ]]; then
    curl -s -f -4 "https://api.clio.one/htopology/v1/?port=${CNODE_PORT}&blockNo=${blockNo}&valency=${CNODE_VALENCY}&magic=${NWMAGIC}${T_HOSTNAME}" | tee -a "${LOG_DIR}"/topologyUpdater_lastresult.json
  fi
  if [[ ${IP_VERSION} = "6" || ${IP_VERSION} = "mix" ]]; then
    curl -s -f -6 "https://api.clio.one/htopology/v1/?port=${CNODE_PORT}&blockNo=${blockNo}&valency=${CNODE_VALENCY}&magic=${NWMAGIC}${T_HOSTNAME}" | tee -a "${LOG_DIR}"/topologyUpdater_lastresult.json
  fi
fi
if [[ ${TU_FETCH} = "Y" ]]; then
  if [[ ${IP_VERSION} = "4" || ${IP_VERSION} = "mix" ]]; then
    curl -s -f -4 -o "${TOPOLOGY}".tmp "https://api.clio.one/htopology/v1/fetch/?max=${MAX_PEERS}&magic=${NWMAGIC}&ipv=${IP_VERSION}"
  else
    curl -s -f -6 -o "${TOPOLOGY}".tmp "https://api.clio.one/htopology/v1/fetch/?max=${MAX_PEERS}&magic=${NWMAGIC}&ipv=${IP_VERSION}"
  fi
  if [[ -n "${CUSTOM_PEERS}" ]]; then
    topo="$(cat "${TOPOLOGY}".tmp)"
    IFS='|' read -ra cpeers <<< "${CUSTOM_PEERS}"
    for cpeer in "${cpeers[@]}"; do
      IFS=',' read -ra cpeer_attr <<< "${cpeer}"
      case ${#cpeer_attr[@]} in
        2) addr="${cpeer_attr[0]}"
           port=${cpeer_attr[1]}
           valency=1 ;;
        3) addr="${cpeer_attr[0]}"
           port=${cpeer_attr[1]}
           valency=${cpeer_attr[2]} ;;
        *) echo "ERROR: Invalid Custom Peer definition '${cpeer}'. Please double check CUSTOM_PEERS definition"
           exit 1 ;;
      esac
      if [[ ${addr} = *.* ]]; then
        ! isValidIPv4 "${addr}" && echo "ERROR: Invalid IPv4 address or hostname '${addr}'. Please check CUSTOM_PEERS definition" && continue
      elif [[ ${addr} = *:* ]]; then
        ! isValidIPv6 "${addr}" && echo "ERROR: Invalid IPv6 address '${addr}'. Please check CUSTOM_PEERS definition" && continue
      fi
      ! isNumber ${port} && echo "ERROR: Invalid port number '${port}'. Please check CUSTOM_PEERS definition" && continue
      ! isNumber ${valency} && echo "ERROR: Invalid valency number '${valency}'. Please check CUSTOM_PEERS definition" && continue
      topo=$(jq '.Producers += [{"addr": $addr, "port": $port|tonumber, "valency": $valency|tonumber}]' --arg addr "${addr}" --arg port ${port} --arg valency ${valency} <<< "${topo}")
    done
    echo "${topo}" | jq -r . >/dev/null 2>&1 && echo "${topo}" > "${TOPOLOGY}".tmp
  fi
  mv "${TOPOLOGY}".tmp "${TOPOLOGY}"
fi
exit 0
10

Maybe check this one as well and see if there might be any differences producing the final curl call to the topologyUpdater api when registering the node:

(The first script registers your node and must be run once every hour to send a heart beat)

11

I worked through the sample script on the above link and I get the same result:

"IP is not (yet) allowed to fetch this list"
12

Please DM me the public IP. I’ll look into later today (+4h from now)

14

@werkof can you please confirm the you received the information via the DM. Thanks in advance for your assistance.

15

Yes received, sorry for the unexpected delay. will dig in and reply asap

16

Issue is fixed at TU server side. Caused by geo position data (region id) of the clients IP.
In over a year and - so far - over 7000 unique served IPs, the first appearance

1 Like
17

Thank you both @werkof and @mcrio for helping me fix my issue.

2 Likes
18

Hello, I think I’m having the same issue as Mick66. My testnet is built and running with a similar topology. The topology updater script log from cron shows:
{ “resultcode”: “204”, “datetime”:“2021-09-06 12:33:06”, “clientIp”: “20.80.54.8 9”, “iptype”: 4, “msg”: “glad you’re staying with us” }
{ “resultcode”: “204”, “datetime”:“2021-09-06 13:33:03”, “clientIp”: “20.80.54.8 9”, “iptype”: 4, “msg”: “glad you’re staying with us” }
{ “resultcode”: “204”, “datetime”:“2021-09-06 14:33:00”, “clientIp”: “20.80.54.8 9”, “iptype”: 4, “msg”: “glad you’re staying with us” }

The BP and Relay nodes are processing transactions, however my outbound connections are only limited to a handful of hosts. Incoming connections seem to be populated (10 incoming connections.) However it seems the outgoing connections are limited (3). I’m receiving the same error as stated in this post. Any help is appreciated!

However when trying to pull the relay list it has the same error as his…ip included:
{ “resultcode”: “402”, “datetime”:“2021-09-06 15:01:23”, “clientIp”: “20.80.54.89”, “iptype”: 4, “msg”: “IP is not (yet) allowed to fetch this list”

19

This probably is because every network other than mainnet requires the magic parameter specified in the fetch URL. For example

https://api.clio.one/htopology/v1/fetch/?magic=1097911063

PLease try this ^^ and let me know…

1 Like
20

That worked! Thanks for the quick response!

1 Like
21

Yo, getting the same issue as previous users:

“iptype”: 4, “msg”: “IP is not (yet) allowed to fetch this list”