Actually the memory usage grows over time as the node runs in the background until it runs out (thus using swap space / virtual memory on disk). It doesn’t matter it you have 8, 16, or 32 GB of ram as it will burn through it all very quickly … this is indicative of a rampant memory leak somewhere. Restarting the machine or node periodically is the only work around I have found thus far.
Looks like Raph made a similar python script based on timing the response by actually connecting a socket to each node rather than just using a port scan to see if they are available. He put a 3 second timeout on it then sorts the results after by the diffs, if successful. That works too although a bit slower to run. I like his inclusion of max_num_rows as well since I found the more valency and connections the faster the memory gets eaten up!
I actually modified the previous workaround script a tad this weekend to include state filtering and a shorter 250ms timeout. As Raph mentioned ICMP is almost always blocked by firewall so you got to do something a tad more creative than a ping to see if there node is good. I think I will add a max connection ratio based on system memory for the final touch.
It will be much better when they fix connectivity and discovery. Regular users are not going to write code, dig through log files, and conduct their own workaround research. They are going to say “this is broken” and go somewhere else that doesn’t have fundamental usability issues.
#!/bin/python3
import json
import requests
# TODO: SET YOUR REGION HERE!
# e.g. 'North America', 'Europe', 'Asia', 'Africa', 'South America', 'Oceania'
region = 'North America'
# TODO: SET YOUR STATES HERE! (optional)
# e.g. ['Arizona', 'California', 'Nevada', 'Utah', 'Colorado', 'New Mexico']
# e.g. ['FR', 'GB', 'BE', 'DE', 'CH', 'IT', 'ES']
states = ['Arizona']
# TODO: SET MAXIMUM NODES TO ADD HERE!
max_nodes = 4
# collect relay nodes
cardano_nodes = []
# port check courtesy of https://www.yougetsignal.com/tools/open-ports/
uri = 'https://ports.yougetsignal.com/check-port.php'
# helper to add responsive nodes
def transform_node(node):
keys = ['addr', 'port']
res = {x:node[x] for x in keys}
res['valency'] = 1
cardano_nodes.append(res)
def main():
# fetch latest relay nodes from Cardano explorer
response = requests.get('https://explorer.cardano-mainnet.iohk.io/relays/topology.json')
if response.ok:
body = json.loads(response.text)
# filter local region nodes
nodes = (n for n in body['Producers'] if n['continent'] == region)
# optional filter by states as well
if states:
nodes = (n for n in nodes if n['state'] in states)
for node in nodes:
port = node['port']
addr = node['addr']
success = 'is open on ' + addr
raw = {'remoteAddress': addr, 'portNumber': port}
try:
# check if node is available
yougetsignal = requests.post(uri, data = raw, timeout=0.25)
if yougetsignal.ok:
print(yougetsignal.text)
if success in yougetsignal.text:
print('Adding ', node)
transform_node(node)
if len(cardano_nodes) == max_nodes:
break;
else:
print('Skipping ', node)
else:
print('Request failed for YouGetSignal!')
except Exception as ex:
print('Request timeout for ', node)
# always add IOHK default relays
cardano_nodes.append({'addr':'relays-new.cardano-mainnet.iohk.io','port':3001,'valency':1})
topo = {'Producers': cardano_nodes}
with open('topology.yaml', 'w') as out:
out.write(json.dumps(topo))
else:
print('Request failed for Cardano explorer!')
if __name__ == '__main__':
main()
Edit 2: For giggles I ran the new script to make sure I didn’t have any typos before posting and then dumped the topo file into Daedalus. Just had a record breaking startup and sync in under two minutes and that has never happened before! Completely unfair test as I had previously messed around earlier today but it’s the little things …