Cardano-node + cardano-db-sync

Developers Cardano Integration
1

I have cardano-node and cardano-db-sync built from source running on my dev machine successfully but now I’m trying to duplicate the setup on what will be the production server.

I’m using the prebuilt binaries from https://hydra.iohk.io but now running into all sorts of problems. For example:
[db-sync-node:Error:3239] [2021-10-07 13:19:39.53 UTC] recvMsgRollForward: AsyncCancelled
[db-sync-node:Error:3236] [2021-10-07 13:19:39.53 UTC] runDBThread: AsyncCancelled
[db-sync-node:Error:3231] [2021-10-07 13:19:39.53 UTC] ChainSyncWithBlocksPtcl: AsyncCancelled
[db-sync-node.Subscription:Error:3227] [2021-10-07 13:19:39.53 UTC] Identity Application Exception: LocalAddress “/home/ubuntu/cardano/mainnet/db/node.socket” MuxError MuxBearerClosed “<socket: 21> closed when reading data, waiting on next header True”

cardano-node seems to be running fine but cardano-db-sync is getting a lot of errors. I’m running cardano-node 1.30.0 and cardano-db-sync 11.0.4 Are these two versions incompatible?

If someone can point me to the current versions and configuration of cardano-node and cardano-db-sync that are compatible with mainnet and testnet it would be greatly appreciated.

2

The latest stable version for the cardano node is 1.30.1, previous was 1.29.0, but not sure if that is related.

1 Like
3

Possibly. So far I haven’t been able to find prebuilt binaries of 1.30.1. Do you know if these have been released yet?

4

Sorry, not sure. I’m building those manually.

5

The binaries should always be available, I believe they’re automatically built. Here’s a snippet from discord during a discussion I was having earlier with @georgem1976

George [APEX] — Today at 9:03 AM
you have time for breakfast. it takes about an hour.
[9:03 AM]
and btw, you don’t have to build it if you don’t want. you can use the binaries from IOG.

Dilworth — Today at 9:05 AM
I didn’t realise the binaries for 1.30.1 were available? Ah well, it’s building now; yeah I think I need some food Thanks

@Dilworth
I didn’t realise the binaries for 1.30.1 were available? Ah well, it’s building now; yeah I think I need some food Thanks

George [APEX] — Today at 9:23 AM
they are always available. you can find the link on the github releases page, Hydra binaries. For the 1.30.1 you have to expand “Downloads” to see the link.

Hopefully, that is of some help.

2 Likes
6

Thank you so much. Still new to this ecosystem. It’s very logical, but sometimes not obvious.

7

Just wanted to follow up. I ended up building from source. These versions are working well together on mainnet and testnet:

$ cardano-cli --version
cardano-cli 1.30.1 - linux-x86_64 - ghc-8.10
git rev 0fb43f4e3da8b225f4f86557aed90a183981a64f

$ cardano-db-sync --version
cardano-db-sync 11.0.4 - linux-x86_64 - ghc-8.10
git revision b5cd53d2e86f45127d98de429e8a2b6879fd8a56

8

@kris.henderson76 : Out of curiosity, what kind of machines do you use for that job ?
We use a 64 GB Ram, 8 Core machine with 2TB HDD.

In the newest release the node v1.30.1 takes up to 12 GB of ram.
The cardano-db-sync also can take something in between 8 - 12 GB ram.
The Postgres db is relatively friendly in terms of ram, but due to the large db size (120Gb on our side at the moment) we discovered also issues with less than 4 GB Ram. To support a high concurrency scenario and multiple connections, we increased the possible ram usage for postgres to 16GB.
To be setup for a next release (which might use again more ram), we decided to stick with such a heavy machine.

Whats your take on that ?

9

I was running on a machine with 16GB RAM and 512GB SSD. I quickly outgrew that and came to the same conclusion as you, upgrading to 64GB RAM and 2TB SSD. At the moment, 6 CPU cores is performing well for me though.

10

Thanks for that info.

Do you use the restore option or do you let it sync from zero ?
I had to resync two weeks ago due to an issue with the setup.
Unfortunately the last 10% from 90 - 100 are a pain in the ass. Even for this big machine it takes one day to sync 4 blockchain days. Do you have the same behavior ?

https://mainnet.cutymals.com/api/CardanoSyncStatus?X-API-KEY=ILoveCutyMals

{
  "syncedTimestamp": "2021-11-24T22:53:47",
  "syncedInPercent": 98.614445915784
}
11

hi @tigrpool.com did u find a better way, can we do it directly without syncing somehow?

12

Hello @AkshitV,
you will achieve the fastes db sync with using the RESTORE_SNAPSHORT=https://....schema...x86_64.tgz option in your docker file.
It takes about 2 hours to download and restore the whole database. Unfortunately i once had the issue that an already “restored database” was dropped and recreated after a restart…

I went with providing two servers, one which is live and one which is the backup for mainnet.cutymals.com.
Try not to restart the cardano-db-sync at all, is my tip (atleast for v11.x version).
When i restart the cardano-db-sync on a fully synced server (where db is >= 99.9 %), cardano-db-sync starts deleting one week of data and then resyncs that deleted data. This takes 3-4 days till the server is fully operational again and showing live data.

So if i have to do adjustments, i try not to restart the db-sync and if i have to i take one of the two servers offline, let it sync and then switch it with the live node.

Hint: The testnet node is no comparison to that. Testnet node syncs withing 1-2 days and is then fully operational. Sync on the mainnet takes more than 2 weeks.

As we provide our API for people to get around such issues, i want to point out it’s also free to use.
If you miss something or you need adjustments for implementing your use-case feel free to write a pm, i am happy to support your use-case.