Attention SPOs - 1.24.2 UPGRADE GUIDE - Upgrade NOW for the upcoming Allegra Hard Fork Event!

Staking & Delegation Operate a Stake Pool
1

Hello SPOs!!!

It is recommended all nodes be updated to version 1.24.2 ASAP in preparation for the upcoming HFE (hard for event) on December 16, 2020. Nodes that are not updated after the chain transitions will not mint blocks!!!

Also, as a reminder - PLEASE CHECK YOUR KES KEY EXPIRATION - many are coming due and will need to be rotated. A block producing node with expired KES keys will NOT be able to mint!

check kes expiration from your block producer:

curl localhost:12798/metrics | grep -i forge

and look for the following line item:

cardano_node_Forge_metrics_remainingKESPeriods_int xx

the “xx” integer displayed reflects the remaining valid KES periods (1 KES period = 1.5 days)

1.24.2 Upgrade Guide provided on behalf of SPOCRA

first let’s update our linux packages on our nodes:

sudo apt-get update
sudo apt-get upgrade -y

now let’s see if ghcup is installed:

ghcup --version

install ghcup if needed:

curl --proto ‘=https’ --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh

answer “NO” to:
“Do you want to install haskell-language-server (HLS) now?”

type “NO” and press Enter

answer “YES” to:
"If you want ghcup to automatically add the required PATH variable to "/home/[user]/.bashrc

type “YES” and press Enter

if by chance you receive the following error: “sh: 277: cannot create /home/[user]/.bashrc: Permission denied”, you will need to manually update your .bashrc and add the following:

[ -f “/home/[user]/.ghcup/env” ] && source “/home/[user]/.ghcup/env” # ghcup-env

now source your .bashrc:

source .bashrc

confirm ghcup is installed:

ghcup --version

make sure ghcup is on the latest version; you may need to run more than once (current version is v0.1.12):

ghcup upgrade

*note - if ghcup --version outputs v0.1.11, this can be remedied via running the following again:

curl --proto ‘=https’ --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh

install ghc 8.10.2:

ghcup install ghc 8.10.2

set ghc 8.10.2 as the default version:

ghcup set ghc 8.10.2

verify version 8.10.2 is set correctly:

ghc --version

// The Glorious Glasgow Haskell Compilation System, version 8.10.2
// yay!

Now we need to ensure proper write protections on block producer hot keys (KES, VRF and opcert) - make sure they are read only. This will be required if upgrading from 1.21.1 (and already in place if upgrading from a later version). You can perform a simple chmod 400 xxx where xxx is the name of your key file to set the file options right. Your core-node will not start up without proper write protection on hot key files! chmod 600 xxx is ok too.

navigate to the folder containing your block producer’s hot keys and set appropriate file permissions:

chmod 400 vrf.skey
chmod 400 kes.skey
chmod 400 opcert

now we need to update our mainnet-config.json file - per the release notes, “Stake Pool Operators (SPOs) and Exchanges should update their node config “options” section with an extra entry:”

“options”: {
“mapBackends”: {
“cardano.node.resources”: [
“EKGViewBK”
],

given there are subtle differences, SPOCRA recommends pulling the new config file from the code-locked build repo:

navigate to the folder containing your config file and rename your existing config file:

mv mainnet-config.json mainnet-config-old.json

pull in the latest config file:

wget https://hydra.iohk.io/build/5102327/download/1/mainnet-config.json

// in the future you can find the latest config file via https://hydra.iohk.io/job/Cardano/iohk-nix/cardano-deployment/latest-finished/download/1/mainnet-config.json

reference your mainnet-config-old.json and update your node logging preferences in your new mainnet-config.json

remove your mainnet-config-old.json:

rm mainnet-config-old.json

for better block visibility performance, add the following line to the mainnet-config.json just for your relay node(s):

“MaxConcurrencyDeadline”:2,

On setups with more networking resources / bandwidth available it is recommended to set MaxConcurrencyDeadline to 3 or 4 on relays and 2 on block producers

The MaxConcurrencyDeadline parameter tells your relays how many duplicate blocks to pull in to help ensure the relay propagates correct blocks. The higher the value for MaxConcurrencyDeadline, the more resources the node will utilize (default is 1)

now we update our nodes to cardano-node version 1.24.2:

cd
cd cardano-node
cabal clean
cabal update
git fetch --all --tags
git checkout tags/1.24.2
cabal build all

[stop your node]

copy over binaries - this assumes cardano-node is located in ~/.local/bin/ (update for cardano-node in ~/.cabal/bin/ as necessary):

cp -p dist-newstyle/build/x86_64-linux/ghc-8.10.2/cardano-node-1.24.2/x/cardano-node/build/cardano-node/cardano-node ~/.local/bin/

cp -p dist-newstyle/build/x86_64-linux/ghc-8.10.2/cardano-cli-1.24.2/x/cardano-cli/build/cardano-cli/cardano-cli ~/.local/bin/

[start your node]

confirm the correct version is installed 1.24.2:

cardano-node --version

verify your node is up and on tip!

cardano-cli query tip --mainnet

*notice we removed the “shelley” keyword from the above command!

Please post any questions and/or feedback! Thank you and happy updating!

Your Friend, FROG

20 Likes
2

Thanks for sharing Kyle! :slight_smile:

A little thing I noticed which might save others from an error message:

In case you want to modify your existing config-file use caution when copy&pasting the new options/mapBackends/cardano.node.resources/EKGViewBK from the posting above.
The forum software here converted the quotes into text-quotes! So be sure to use real " instead in your config file

2 Likes
3

thanks @ADAfrog - this is a great help to the community :heart_eyes:

One question please, since everything else in this nice checklist is documented one place or another: where did you find out about MaxConcurrencyDeadline? Given that it’s not in the latest official config file, is there anywhere we can find out more about this parameter before we start tuning it on our servers? Anyone else is welcome to respond as well :sunglasses: :pray:

1 Like
4

We picked it up as Pioneer during HTN. I have no clue where it may be documented. I have been running with it since shelley launch.

The general idea is the higher the value, the less likely your node will be to mint blocks on forks. You will need to characterize the effects on your nodes - it mostly affects bandwidth.

3 Likes
5

Superb guide!! Thanks Kyle!

2 Likes
6
4 Likes
7

Thanks for the guide and notice. Our pool has already upgraded to 1.24.2, however I had to ask research when the hard fork would take place. There was no clear announcement on this, at least on the channels I was on.

I’m subscribed to the announcements channel on Telegram it would be great if mandatory node upgrades or releases are also mentioned there.

8

Do you think its still needed given the link relates to byron era? Curious given IOHK no longer add it into the base config or even mention?

1 Like
9

There’s been a suggestion from the CF Team (@adatainment) to get all these releases & operational announcements in one place on the forum…

10

Thanks - odd the default is two? We were advised from Sam default was one.

I’ll follow up to see if we can learn more / confirm here to understand to what extent the param is relevant.

1 Like
11

Thank you! I appreciate you, Lauris!

1 Like
12

Another suggestion, based on what we noticed when installing 1.23.0, is to check the logfile immediately after restarting the new node for the first time. If permissions are too loose on any of these files & upgrading from an older version than that, the node will not fully start & messages like this will display repeatedly, at the very beginning with no timestamp:

VRF private key file at: ./config/vrf.skey has "other" file permissions. Please remove all "other" file permissions.

Also, due to UNIX demand paging which maps the running process to where it really lives on the disk, there is no need to stop the old node before copying the new node:

  • The active cardano-node can be mv'd to a new name and it will continue to run while copying the new binaries into place.
  • This always saves a little bit of node downtime, but sometimes a lot in case operators have trouble with the long pathnames.
1 Like
14

Yes, thank you for this. Using mv can help shave a few seconds

1 Like
15

Hi all,

I hope this finds you well. Thank you @ADAfrog for the post and to all contributors. This is very informative.

I have a question: I updated both ghc and the node software a few days ago, without ghcup. I had an original directory with ghc 8.6 and created a new one with ghc 8.10. This resulted in manually removing a number of things. The link to the new binary was updated automatically. Would having ghcup installed help with overwriting existing versions of ghc?

Cheers,

A

16

HI,

Is there a way for me to get or compile binaries for aarch64?

17

Nice Job! !!

1 Like
18

Doesn’t like the --proto ‘=https’ on some nodes.

19

Great thanks for the notes, really cool that someone put them out there. Saved me a ton of time rebuilding the whole thing. Should add that end-to-end upgrade took about 1 hour.

I should say though that these sort of upgrades that risk the node to stop producing blocks should really be better advertised… a big banner in on the main page of the forum, that sort of thing.

20

Your example above is using typographical single quotes (you can tell since the left & right ones look different) so if copy & pasting then they’ll be passed along with the argument itself & the argument won’t be valid.

If that is your problem it’s the same issue as comment #2 above. These scripts have to be posted as code, not quotes, in order to be copy & paste-able. :sunglasses:

2 Likes
21

Nice catch, that was the issue. Typed it a few times and then copied it and it wasn’t happening.

1 Like