No Space Left on Device shortly after Running Relay

Problem

The cardano-node relay stops running after about 15min, and can no longer SSH into the VM (the serial port log on the VM in GCP says ‘not enough disk space’ or no space left)

My machine is a e2-medium (2 vCPU, 4GB memory) w/ 10GB SSD VM hosted as a compute engine in GCP. I’m following instructions from the Stake Pool Course.

Here’s the log right when it happened:

Debugging & Questions

command used:

cardano-node run \
 --topology testnet-topology.json \
 --database-path db \
 --socket-path db/node.socket \
 --host-addr 0.0.0.0 \
 --port 3001 \
 --config testnet-config.json
  1. When I do various ls and du commands to check the size of my ~/cardano-node/db directory, its contents don’t look that large (the .dat files), does this imply that maybe it’s a file that’s not in ~/cardano-node/db that is consuming too much disk space? Log file hanging around somewhere?

  2. Here is the output from running htop, why is cardano-node listed so many times, that normal? 1 instance of it running but using or sharing more more CPU’s?

  3. I run the commands tree and df to observe memory and disk consumption, any key metrics to look for when running those? When I ran ‘tree’ I noticed the “available memory” was slowly dropping over time.

I captured this log right after a fresh cardano-node install but before running a node.
before-running-cardano-node

  1. Any params I can tweak? Flags passed to cardano-node run. I’m checking them now.

  2. Need to update the testnet-config.json at all?

I have fresh install backup up via machine image and snapshot so can iterate on any suggestions! Thanks

Hello,

10Gb ssd it’s not enough, you will need more, at least 40gb to be safe.

You should check the size of ledger folder, logs folder, etc

Minimum requirements for running a stake pool

In terms of hardware, you should have the following available:

  • 4 GB of RAM
  • 24 GB of hard disk space
  • a good network connection and about 1 GB of bandwidth per hour
  • a public IP4 address

Note that processor speed is not a significant factor for running a stake pool.

Cheers,

1 Like

Thanks Alexd1985 I will try that.

Does a relay node download the entire blockchain? Is it validating transactions?

I think since I’m running cardano-node with --database-path db the answer is Yes.

I believe it does, but it’s hard to tell from these docs:
https://docs.cardano.org/en/latest/getting-started/stake-pool-operators/index.html.

https://docs.cardano.org/projects/cardano-db-sync/en/latest/getting-started/validation.html?highlight=blockchain%20history#validation

https://docs.cardano.org/en/latest/explore-cardano/cardano-architecture-overview/index.html

most docs seem to not distinguish if it’s a Relay or Core/BP node. I understand it’s still the same 'cardano-node` program.

I will start at +24GB.
https://cardano-foundation.gitbook.io/stake-pool-course/stake-pool-guide/system-setup/aws recommends 24GB

https://cardano-foundation.gitbook.io/stake-pool-course/stake-pool-guide/getting-started/install-node recommends 10GB

https://docs.cardano.org/projects/cardano-node/en/latest/getting-started/install.html recommends 10GB

Will

Relay or BP are the same, downloading all blockchain, validating transaction, etc

The only difference is that only the BP can create blocks…

1 Like