HOW TO: Running Cardano as a service AND still being able to Debug with journalctl in UBUNTU

Hi guys,
I don’t know how you guys check the node log when running as a service, but cardano-service status isn’t a good way in my opinion.

EDIT: This was only tested on Ubuntu.

Okay so first of all I made the node run as a service following these steps:

  1. Create the start file the systemd service will be executing
    cd .local/bin
    sudo nano cardano-service

  2. Add this in the file for Relays WARNING: Pay attention to the paths and especially DB location, if db is in another location you can create a new path variable resembling FILES.

#!/bin/bash
FILES=/home/[USER]/PATH-TO/FILES
PORT=3000
HOSTADDR=0.0.0.0
TOPOLOGY=${FILES}/mainnet-topology.json
DB_PATH=${FILES}/db
SOCKET_PATH=${FILES}/db/socket
CONFIG=${FILES}/mainnet-config.json

cardano-node run \
--topology ${TOPOLOGY} \
--database-path ${DB_PATH} 
--socket-path ${SOCKET_PATH} \
--host-addr ${HOSTADDR} \
--port ${PORT} \
--config ${CONFIG}
  1. For your Block Producer use this and make sure the certificate and key paths are correct.
#!/bin/bash
FILES=/home/[USER]/PATH-TO/FILES
PORT=3000
HOSTADDR=0.0.0.0
TOPOLOGY=${FILES}/mainnet-topology.json
DB_PATH=${FILES}/db
SOCKET_PATH=${FILES}/db/socket
CONFIG=${FILES}/mainnet-config.json
KEYS=${FILES}/keys
KES_KEY=${KEYS}/kes.skey
VRF_KEY=${KEYS}/vrf.skey
CERTIFICATE=${KEYS}/certificates/node.cert

cardano-node run \
  --topology ${TOPOLOGY} \
  --database-path ${DB_PATH} \
  --socket-path ${SOCKET_PATH} \
  --host-addr ${HOSTADDR} \
  --port ${PORT} \
  --config ${CONFIG} \
  --shelley-kes-key ${KES_KEY} \
  --shelley-vrf-key ${VRF_KEY} \
  --shelley-operational-certificate ${CERTIFICATE}
  1. Make sure you allow this file to be executed
    sudo chmod +x cardano-service

  2. Test if the node starts properly first by executing this in a tmux window
    ./cardano-service

  3. If it works properly leave that tmux window (CTRL+B D) and stop the node (so it starts faster next time)
    killall -s SIGINT cardano-node

  4. Create the service file
    sudo nano /etc/systemd/system/cardano-node.service

  5. Paste (RClick) this content and edit the paths

# The Cardano Node Service (part of systemd)
# file: /etc/systemd/system/cardano-node.service 

[Unit]
Description     = Cardano node service
Wants           = network-online.target
After           = network-online.target

[Service]
User            = [USER RUNNING THE NODE:] ex: ada
Type            = simple
WorkingDirectory= /home/[USER]/PATH-TO-NODE-FOLDER
ExecStart       = /bin/bash -c "PATH=/home/[USER]/.local/bin:$PATH exec /home/USER/.local/bin/cardano-service"
KillSignal=SIGINT
RestartKillSignal=SIGINT
TimeoutStopSec=3
LimitNOFILE=32768
Restart=always
RestartSec=5

[Install]
WantedBy= multi-user.target
  1. Save the file and reload Daemons
    sudo systemctl daemon-reload

  2. To avoid needing to call sudo systemctl we can edit .bashrc
    sudo nano $HOME/.bashrc
    and add the following lines

cardano-service() {
    sudo systemctl "$1" cardano-node.service
}
  1. Enable the changes made
    source $HOME/.bashrc

  2. Enable and start the service NOTE: execute these commands one by one to avoid problems

cd $HOME
cardano-service enable
cardano-service start
cardano-service status
  1. You can get a view from what your node is doing with cardano-service status, but I advise to use journalctl

(You can download ccze for colors)
sudo apt install ccze
journalctl -f --output=cat -u cardano-node.service | ccze -A

-f for following live
–output=cat to remove user and timestamps
-u to only follow cardano-node.service

  1. You can put this in a file like for example monitor.sh, chmod +x it and ./ it everytime you want to monitor/debug stuff from your node.

  2. Don’t forget you can change minSeverity: Info in mainnet-config.json, the options from more to less notifications are: Debug , Info , Notice and Warning and Error

I hope I could help somebody with this and good luck!

P.S. For edits/improvements let me know!
Grtz Mex

Sources:
https://docs.cardano.org/projects/cardano-node/en/latest/logging-monitoring/logging.html

If you like to run on a RaspberryPi4, it can actually be much simpler.

I read some posts a couple of months ago that advised against running a cardano stake pool in docker, it would supposedly compromise security.
Care to elaborate on that ?

And the easiest way to run a Raspberry node was to use Allessandro’s custom aarch64 repo. But ofcourse experiences may differ.

Personally my bp runs on a normal server PC and Relays on different networks at family members on a raspberry.

Thanks for the addition.

Great!
Nice idea for the bash functions! will implement some, thanks!

for the systemd I’m using a builtin EnvironmentFile option

[Unit]
Description=Cardano BP Node
After=multi-user.target

[Service]
Type=simple
EnvironmentFile=/home/cardano/cnode/config/cardano-node.environment
ExecStart=/home/cardano/.local/bin/cardano-node run --config $CONFIG --topology $TOPOLOGY --database-path $DBPATH --socket-path $SOCKETPATH --host-addr $HOSTADDR --port $PORT
KillSignal = SIGINT
RestartKillSignal = SIGINT
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=cardano
LimitNOFILE=32768

Restart=on-failure
RestartSec=15s
WorkingDirectory=~
User=cardano
Group=cardano

[Install]
WantedBy=multi-user.target

and then just add the variables in the env file:

CONFIG="xxx"
TOPOLOGY="xxx"
DBPATH="xxx"
SOCKETPATH="xxx"
HOSTADDR="xxx"
PORT="xxx"
1 Like

Nice! Can you explain why you would seperate the EnvironmentFile?

Thanks!
Mex

If the point is to run on arm architecture (aarch64 as @Mextro was pointing out) you can just use real arm based servers. Either cloud based or DC based would do, but something a tad more serious than a raspberry is probably a good idea.
I distribute the compiled binaries regularly here and soon on https://www.myadanode.com with tutorials.

On the raspberry point, I can’t see the benefit from a cardano network point of view ?

  • It’s cheaper for sure, but cardano reward you for a reason right. Having stable and reliable node IS important ! I’m sorry to say but as much as I love raspberry I would not sit a multi billion economy on it.
  • It’s fun. Yes for you as a project probably. But from the cardano point of view it’s just another arm based pool running.
  • You are running them home, so you can eventually be impacted by electricity outage (it happens to all of us), internet issues (same), human and life events.

Happy to be proven wrong here, but it would need some serious arguments. And yeah I guess relays can be an option … i guess …

Thanks mate for the tutorial, I have automated the exact same but I still learned 1 thing from it. So just for this, thank you :wink:

Would running nodes for six months without hickup, 32 blocks, 137% luck, 2454 ADA payout of pool rewards that prefer to give back to delegators be good enough for a reason?

I understand your argument and why it is a strong one in your opinion.
I’m sorry but that doesn’t address the point I was mentioning. Where are those raspberry? How is internet delivered to them? What about power? What if you need more than 8g ram in a near future? The list goes on and I think you get my point.
All this might not stop you from being successfull, but if I had to bet on pool stability and efficiency I definitely would place my bet on a deployement that limit the risks.
I have been doing this professionnaly for 20+ years so I got so insight and experience. But don’t take that arguement as a “I know better”. I just want to mention that I can back my arguments too.

For the money side, I think a pool deserve reward. As any useful work does. Risk again, chances of getting volunteers doing it for free for the next 20+ years (and more) at a professional quality to help a global economy unrolling and permitting other people to earn money is … thin … unrealistic …
but again, we all got our ways, future will tell.

Well, I’m sure you have plenty of experience with hosted arm64 devices and know all about the utilisation metrics of those 4 core machines.

Yes, I agree every pool can of course have whatever business model they like. In case you’d like to know more about this particular model i.e. how the loyalty payout system works, I’m happy to talk you through the motivation, where I think it might lead and how it worked out so far.

@kaverne @tomdx
First of all, I want to clarify this tutorial is meant for ALL Ubuntu Nodes, not just PI nodes, the source listed is just because that was the easiest and clearest way to add a systemd service for a complete tutorial.
Secondly: running a stakepool requires the time, skill and interest in servers to build it from scratch right? Keeping that in mind the pool also requires maintenance like updating to the newest version and renewing the KES.

Considering weekly checkups are to be done, adding more relays on different devices isn’t a bad idea, and shouldn’t be that time consuming after doing the initial setup.

The whole point of crypto is decentralization. So whether you have 20 year server experience or you’re just getting here like me, decentralized political/financial networks are brand new (2009).
This isn’t just some peer to peer file sharing service, we can surely agree on that.

So decentralization purism is required to a degree if we want to make this project a succes. That being said, hosting your pool at AWS or some other hosting service isn’t that decentralized right? Sure the online times and security are some pro’s… But in the end what matters to you most, monetary revenue or supporting one of the greatest technological advancements of the 21st century?

Personally I want to grow my stakepool and attract more stake and promote Cardano to new users, family and friends in the process, finally giving them a fair way of getting interest on their money.

I have 3 servers running as mentioned before, 2 PI’s at the houses of family members, low power usage, low noise, low heat ouput, automatic restart after power outage and low cost all added to the attractiveness of getting a PI. Which in my opinion is a great piece of technology and not just a mere toy. That being said, if more stake pools are to be run by ‘normal’ people in the future, a pi would make good hardware for that, not all people have space or need for bigger machines, and hosting services aren’t decentralized. If Amazon becomes anti Cardano (high chance they will, you can’t imagine the financial impact Cardano will make in the next 20 years) they can just dump 50% of the stakepools easily.
My BP is on another server I built specifically for staking, all nodes are on 3 different networks, kept online by my family and me.
Adding more relays to the network only makes it more decentralized and for 10W that isn’t that much asked right?

A large portion of the cardano network is run on Raspberry Pi’s, even the founder of Cardano Charles Hoskinson runs his pool on a Raspberry Pi

Cardano is open to all, join us and help change the world.

Grtz Mextro
[VRIJ] Vrijstaat (Dutch Freestate) Stakepool
www.vrijstaat.net

Yes absolutely it’s open to all and different take on the problem are anyway interesting. It was just a mere chat about opinions and differences.

Well, I’m sure you have plenty of experience with hosted arm64 devices and know all about the utilisation metrics of those 4 core machines.

I sense sarcasm here, If I got you upset @tomdx it wasn’t my goal. I was just trying to get a different point of view by expressing mine.
I’m well aware of you business model since I have checked it few weeks ago.

About AWS, there is not way they would forbid you to do cardano and anything else on their dataccenters @Mextro . First because they don’t access you data, that would be the end of their business, second because they are making money providing a service. And this is their business model, offering services you can consume for a price.
There is currently 81 DC over 25 regions in the world, is that decentralized enough ? Plus people can the use Azure too, for another set of DC over some more regions. The cardano foundation themself are using AWS (and cloud to an extend) as a good hosting way. It surely is not the only good solution, but it is definitely one.

The choice is security and stability VS price and hmmm well I’m short here … since Cloud arm64 will not chew more energy, if anything it will be less by resource sharing.

Again all opinions are good, there is no wrong one, I like having those chats since you can always learn from it.