Problems Setting up Block Producer

LivingStory · 23 April 2022 16:51

I have my relay node all setup I believe.
I have created my air gapped machine and created the KES keys, but my pool node still says they still node to be rotated.
and gLiveView shows starting no matter how long I let it run. I have my topology file set on my BP for the private address of my relay, and the proper port. I have temporarily disabled ufw firewall entirely although this should be unnecessary as I have opened the proper ports on both sender and receiver, even though opening the port on the sender should also be unnecessary.

CNTools still shows my KES keys as needed to be rotated, so maybe I placed the keys in the wrong location.

** WARNING **
Pool FreeADA in need of KES key rotation
Keys expired! : 19:01:37 ago

press any key to proceed ..

But I am stumped the most on why gLiveView won’t show a good status for my Block Producer. Here is the output of systemctl status cnode.service

sudo] password for s1: 
● cnode.service - Cardano Node
     Loaded: loaded (/etc/systemd/system/cnode.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-04-23 11:32:00 CDT; 13min ago
   Main PID: 1433257 (bash)
      Tasks: 18 (limit: 38291)
     Memory: 716.7M
     CGroup: /system.slice/cnode.service
             ├─1433257 bash /opt/cardano/cnode/scripts/cnode.sh
             └─1433755 /home/s1/.cabal/bin/cardano-node +RTS -N4 -RTS run --topology /opt/cardano/cnode/files/topol>

Apr 23 11:32:00 s1-desktop systemd[1]: Started Cardano Node.
Apr 23 11:32:00 s1-desktop cnode[1433257]: WARN: A prior running Cardano node was not cleanly shutdown, socket file>
Apr 23 11:32:01 s1-desktop cnode[1433755]: Listening on http://127.0.0.1:12798

I have checked on pools.io and I cannot see my TICKER of TFSF, cntools says my pool is registered and my metadata.json file is publicly available

etadata
  Name                : TFSF
  Ticker              : TFSF
  Homepage            : https://thefreeada.com/
  Description         : The Free Speech Forum Cardano Staking Pool
  URL                 : https://thefreeada.com/tfsf.json

One note, as a test ran mv db db-old and reran everything and let it sync again, gLiveView looked normal. It was updating as the node synced again and moved through the epochs to the current one. I did a reboot and then gLiveView looks like the above screenshot. I have systemctl setup to start the node on system startup. Don’t know if this is relevant detail to diagnosing the issue. Thank you in advance for your help.

NOTE:
Note sure how to fix this, but I just ran

cardano-cli query tip --mainnet
{
    "era": "Shelley",
    "syncProgress": "66.92",
    "hash": "c94a711660a7e3a42cb1ddd7ee329a636a66e8a74cb2075b96eed91ec52f7832",
    "epoch": 223,
    "slot": 11350911,
    "block": 4829495
}

So the node is not synced, but when I rerun this command minutes later, it isn’t updating. Even though systemctl says the node is running.

Alexd1985 · 23 April 2022 16:59

You will see the kes expiration warning as long the node is not synced and started as Producer… for the moment ignore the warning…

Now

Regarding the pooltool I see u have an error with metadata hash

https://pool.vet/#0ca8e61d6ce513468766d3a8e5dc04ac111642a09339b4562563c3f3

LivingStory · 23 April 2022 17:07

Thank you for the prompt response! I will work through these, I have to step away from the computer for a bit. But I will work through the links and report back! Thank you again Alex!

LivingStory · 23 April 2022 20:33

How do I change my registered hash to match the hash of the tfsf.json file that is at https://thefreeada.com/tfsf.json?

Alexd1985 · 23 April 2022 20:40

I am receiving error 404 … are u sure the link is ok?

LivingStory · 23 April 2022 20:51

Sorry typo, I have fixed the link. the extended json isn’t available yet, don’t know if that matters yet.

Alexd1985 · 23 April 2022 20:52

Did u modified the metadata after u registered the pool?

Anyway… go to cntools - pool - modify and use the previously parameters configured (just press enter) but when it will ask for the metadata url… after u will insert it should say something… metadata exist… do u wanna use it?

And press yes, then complete the steps and should be ok

Cheers,

LivingStory · 23 April 2022 20:55

That is what I am beginning to suspect. A team member of mine created the file and pushed it to the domain. I sent him the text and told him to copy and paste into a file called tfsf.json and upload it. I am now seeing I should have sent him the file and just had him upload. I can happily repeat the steps, I am just concerned about losing the already registered 500 ADA. If that is not a concern, I will happily repeat the steps.

Alexd1985 · 23 April 2022 20:57

Nope, is just a pool modification which will cost u only the transaction fee ~0,2 ADA for the new certificate… so follow the above steps and it should be ok

And also I saw the pledge will be 50k ADA but u don’t have 50k inside the wallet… yet, u can use an imported wallet for the pledge but then u will need again to go to pool - modify to update the pledge wallet

As an advice, the margin is to high and will not attract delegators (Margin is 7.5%)

Also the link for extended metadata from metadata file is wrong (returning 404) perhaps u didn’t uploaded the file yet?
Cheers,

LivingStory · 23 April 2022 23:52

Yes. After talking with my team member, we were going to set the pledge to 100 ADA, is that a good amount? We were also hoping to start small and just see how this works and set the margin to 0% and just collect the 345 ADA per epoch.
Yes I have not uploaded the extended metadata yet. I am getting ready to modify the pool. I am slightly concerned that for some reason the node isn’t syncing anymore. The output of the below command hasn’t changed in a few hours. Yet the cnode process seems to be running from systemctl status cnode.service output.

s1@s1-desktop:~$ !1957
cardano-cli query tip --mainnet
{
    "era": "Shelley",
    "syncProgress": "66.91",
    "hash": "c94a711660a7e3a42cb1ddd7ee329a636a66e8a74cb2075b96eed91ec52f7832",
    "epoch": 223,
    "slot": 11350911,
    "block": 4829495
}

s1@s1-desktop:~$ sudo systemctl status cnode
[sudo] password for s1:
● cnode.service - Cardano Node
Loaded: loaded (/etc/systemd/system/cnode.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2022-04-23 14:47:00 CDT; 4h 4min ago
Main PID: 1298 (bash)
Tasks: 19 (limit: 38291)
Memory: 1.0G
CGroup: /system.slice/cnode.service
├─1298 bash /opt/cardano/cnode/scripts/cnode.sh
└─2243 /home/s1/.cabal/bin/cardano-node +RTS -N4 -RTS run --topology /opt/cardan>

Apr 23 14:47:00 s1-desktop systemd[1]: Started Cardano Node.
Apr 23 14:47:01 s1-desktop cnode[1298]: WARN: A prior running Cardano node was not cleanly sh>
Apr 23 14:47:01 s1-desktop cnode[2243]: Listening on http://127.0.0.1:12798


cntools output

 >> CNTools v9.0.9 - Mainnet - CONNECTED <<          A Guild Operators collaboration

Main Menu Telegram Announcement / Support channel: Telegram: Contact @guild_operators_official

) Wallet - create, show, remove and protect wallets
) Funds - send, withdraw and delegate
) Pool - pool creation and management
) Transaction - Sign and Submit a cold transaction (hybrid/offline mode)
) Blocks - show core node leader schedule & block production statistics
) Backup - backup & restore of wallet/pool/config

) Refresh - reload home screen content

                                                  Epoch 334 - 1d 21:41:11 until next
 What would you like to do?                                   Node Sync: 59192329 :(

  [w] Wallet 
  [f] Funds 
  [p] Pool 
  [t] Transaction 
  [b] Blocks 
  [z] Backup & Restore 
  [r] Refresh 
  [q] Quit 
```

Alexd1985 · 24 April 2022 00:08

Share the glive output l

Type journalctl -e f -u cnode and check for errors and also type free -m and share the output

LivingStory · 24 April 2022 00:29

Apr 23 14:45:49 s1-desktop systemd[1]: Stopped Cardano Node.
-- Reboot --
Apr 23 14:47:00 s1-desktop systemd[1]: Started Cardano Node.
Apr 23 14:47:01 s1-desktop cnode[1298]: WARN: A prior running Cardano node was not cleanly shutdown, socket file still exists. Cleaning up.
Apr 23 14:47:01 s1-desktop cnode[2243]: Listening on http://127.0.0.1:12798

 free -m
              total        used        free      shared  buff/cache   available
Mem:          32027        1993       24186           4        5847       29683
Swap:          2047           0        2047

Seems like for whatever reason the node isn’t actually syncing with the blockchain. But I can’t figure out why that would be. It also concerns me that gLiveView says I have 6 incoming connections on this one. Since my topology.json file looks like

{
  "Producers": [
    {
      "addr": "xxx.xxx.xxx.xxx",
       "port": xxxxx,
       "valency": 2
      }
   ]
}

LivingStory · 24 April 2022 00:38

Should the port number in topology.json be the same number the gLiveView displays. For now firewall is disabled but I have rules on the relay to allow incoming traffic along both of the ports.

Alexd1985 · 24 April 2022 05:49

Should the port number in topology.json be the same number the gLiveView displays

the port from topology file should be the port from glive for the remote node (ur relay port in this case)

Type journalctl -e -f -u cnode and check for errors; if no errors then aply these steps

cd db folder
ls -l (here u should see more files and 3 folders)
sudo systemctl stop cnode
mv ledger ledger_bkp
mv immutable immutable_bkp
mv volatile volatile_bkp
ls - l
sudo systemctl start cnode
after 10 -15 seconds type sudo systemctl stop cnode
ls -l (now u should have the bkp files and also the new files)
delete the new files (not the bkp-ed ones)
rm -R ledger
rm -R immutable
rm -R volatile
ls -l (now u should have the old files only)
rename back the old files
mv ledger_bkp ledger
mv immutable_bkp immutable
mv volatile_bkp volatile
ls -l
start the node and check glive
sudo systemctl start cnode

Triton-pool · 24 April 2022 06:47

You are not going to collect any ADA per epoch. Make sure you understand that you are awarded with the 340 ( or whatever pool fee you set) only if you were assigned a block and if you successfully created the block.
Statistically with 100 ADA in your pool, you can expect 1 block in roughly 50+ years. You have been warned.

Alexd1985 · 24 April 2022 07:19

Exactly, u will receive rewards only if u will create blocks and the pool will need ~1M ada stake (from u or from delegators) for 1block/epoch average

LivingStory · 25 April 2022 17:43

journalctl -e -f -u cnode output

Apr 23 20:39:44 s1-desktop cnode[13046]: NoDbMarkerAndNotEmpty "/opt/cardano/cnode/db/protocolMagicId"
Apr 23 20:39:44 s1-desktop cnode[13046]: cardano-node: Missing "/opt/cardano/cnode/db/protocolMagicId" but the folder was not empty
Apr 23 20:39:44 s1-desktop systemd[1]: cnode.service: Main process exited, code=exited, status=1/FAILURE
Apr 23 20:39:44 s1-desktop systemd[1]: cnode.service: Failed with result 'exit-code'.
Apr 23 20:39:50 s1-desktop systemd[1]: cnode.service: Scheduled restart job, restart counter is at 11.
Apr 23 20:39:55 s1-desktop systemd[1]: Stopped Cardano Node.
Apr 23 20:39:55 s1-desktop systemd[1]: Started Cardano Node.
Apr 23 20:39:55 s1-desktop cnode[14299]: Listening on http://127.0.0.1:12798

Since the last three lines say it was stopped, started, and now just listening. Does this mean cnode process is healthy?
I walked through those steps. I will report back after a few minutes of gLiveView running.

Alexd1985 · 25 April 2022 17:52

now u should see the Mem RSS slowly increasing on glive

LivingStory · 25 April 2022 18:15

Same behavior as before.

When I look at systemctl status cnode this is the output

s1@s1-desktop:/opt/cardano/cnode/db$ sudo systemctl status cnode
● cnode.service - Cardano Node
     Loaded: loaded (/etc/systemd/system/cnode.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-04-25 12:42:29 CDT; 31min ago
   Main PID: 3942241 (bash)
      Tasks: 19 (limit: 38291)
     Memory: 120.9M
     CGroup: /system.slice/cnode.service
             ├─3942241 bash /opt/cardano/cnode/scripts/cnode.sh
             └─3942688 /home/s1/.cabal/bin/cardano-node +RTS -N4 -RTS run --topology /opt/cardano/cnode/files/topology.json --config /opt/cardano/cnode/files/config.json --database-path /opt/cardano/cnode/db --socket-path /opt/cardano/cnode/sockets/node0.socket --port 10106 --host-addr 0.0.0.0

Apr 25 12:42:29 s1-desktop systemd[1]: Started Cardano Node.
Apr 25 12:42:29 s1-desktop cnode[3942241]: ERROR: You specified 12788 as your EKG port, but it looks like the cardano-node (PID: 3942438 ) is not listening on this port. Please update the config or kill the conflicting process first.
Apr 25 12:42:29 s1-desktop cnode[3942241]: WARN: A prior running Cardano node was not cleanly shutdown, socket file still exists. Cleaning up.
Apr 25 12:42:29 s1-desktop cnode[3942688]: Listening on http://127.0.0.1:12798

ps -elf | grep -i cardano output

1@s1-desktop:/opt/cardano/cnode/db$ ps -elf | grep -i cardano
4 S s1         14617       1  0  80   0 -  2971 do_wai Apr23 ?        00:00:00 bash /opt/cardano/cnode/scripts/submitapi.sh
4 S s1         14701       1  0  80   0 -  2971 do_wai Apr23 ?        00:00:00 bash /opt/cardano/cnode/scripts/ogmios.sh
0 S s1         14737   14617  0  80   0 - 268550109 futex_ Apr23 ?    00:00:00 /home/s1/.cabal/bin/cardano-submit-api --config /opt/cardano/cnode/files/config.json --testnet-magic 764824073 --socket-path /opt/cardano/cnode/sockets/node0.socket --listen-address 127.0.0.1 --port 8090
0 S s1         14817   14701  0  80   0 - 268441024 futex_ Apr23 ?    00:00:00 /home/s1/.cabal/bin/ogmios --node-config /opt/cardano/cnode/files/config.json --node-socket /opt/cardano/cnode/sockets/node0.socket --host 127.0.0.1 --port 1337
4 S s1       3942241       1  0  80   0 -  2971 do_wai 12:42 ?        00:00:00 bash /opt/cardano/cnode/scripts/cnode.sh
4 S s1       3942242       1  0  99  19 -  3152 do_wai 12:42 ?        00:00:02 bash /opt/cardano/cnode/scripts/cncli.sh ptsendtip
4 S s1       3942243       1  0  99  19 -  3151 do_wai 12:42 ?        00:00:02 bash /opt/cardano/cnode/scripts/cncli.sh sync
4 S s1       3942244       1  0  99  19 -  3143 do_wai 12:42 ?        00:00:02 bash /opt/cardano/cnode/scripts/cncli.sh leaderlog
4 S s1       3942245       1  0  99  19 -  3152 do_wai 12:42 ?        00:00:02 bash /opt/cardano/cnode/scripts/cncli.sh ptsendslots
0 S s1       3942688 3942241  0  80   0 - 268756240 futex_ 12:42 ?    00:00:07 /home/s1/.cabal/bin/cardano-node +RTS -N4 -RTS run --topology /opt/cardano/cnode/files/topology.json --config /opt/cardano/cnode/files/config.json --database-path /opt/cardano/cnode/db --socket-path /opt/cardano/cnode/sockets/node0.socket --port 10106 --host-addr 0.0.0.0
4 S s1       3943069       1  0  99  19 -  3156 do_wai 12:42 ?        00:00:02 bash /opt/cardano/cnode/scripts/cncli.sh validate
0 S s1       3943955 3943890  0  80   0 -  3184 wait_w 12:42 pts/1    00:00:07 bash /opt/cardano/cnode/scripts/gLiveView.sh
0 S s1       4070989    6912  0  80   0 -  2260 pipe_r 13:23 pts/0    00:00:00 grep --color=auto -i cardano

What’s next in troubleshooting?

Alexd1985 · 25 April 2022 18:26

Reboot the server, sudo reboot and check again

Topic		Replies	Views
KES keys expired - trouble rotating Operate a Stake Pool	55	1928	29 September 2021
Cnode Relay stuck at "starting" Setup a Stake Pool	7	1272	7 March 2021
GLive View Status "Starting" Setup a Stake Pool	43	1817	1 April 2021
KES - Can't rotate my KES keys Operate a Stake Pool	12	357	14 December 2023
Noob Question: Block Producing/Relay Node Setup Setup a Stake Pool	8	423	10 February 2021

Problems Setting up Block Producer

Related Topics