Trouble connecting cardano-db-sync to cardano-node

English Community Technical Support
1

I have a fully synced node up and running and I’m trying to connect cardano-db-sync to pull data from the chain. I am getting some error messages and I can’t find much info to resolve them.

This is what I get when I start cardano-db-sync with the node running:

[db-sync-node:Error:81] [2021-09-21 22:15:06.56 UTC] Error: insertABOBBoundaryblock hash 47a50064150accb15f33e96d1c3ecac988fbd3d682e45fc54d6f2d89857f05b7

$ PGPASSFILE=config/pgpass-mainnet db-sync-node/bin/cardano-db-sync     --config config/mainnet-config.yaml     --socket-path ../cardano-node/state-node-mainnet/node.socket     --state-dir ledger-state/mainnet     --schema-dir schema/
[db-sync-node:Info:6] [2021-09-21 22:15:04.27 UTC] Schema migration files validated
[db-sync-node:Info:6] [2021-09-21 22:15:04.27 UTC] Running database migrations
[db-sync-node:Info:6] [2021-09-21 22:15:05.84 UTC] Using byron genesis file from: "config/../../cardano-node/configuration/cardano/mainnet-byron-genesis.json"
[db-sync-node:Info:6] [2021-09-21 22:15:05.84 UTC] Using shelley genesis file from: "config/../../cardano-node/configuration/cardano/mainnet-shelley-genesis.json"
[db-sync-node:Info:6] [2021-09-21 22:15:05.84 UTC] Using alonzo genesis file from: "config/../../cardano-node/configuration/cardano/mainnet-alonzo-genesis.json"
[db-sync-node:Info:6] [2021-09-21 22:15:06.14 UTC] NetworkMagic: 764824073
[db-sync-node:Info:6] [2021-09-21 22:15:06.39 UTC] Initial genesis distribution present and correct
[db-sync-node:Info:6] [2021-09-21 22:15:06.39 UTC] Total genesis supply of Ada: 31112484745.000000
[db-sync-node:Info:6] [2021-09-21 22:15:06.39 UTC] Inserting Shelley Genesis distribution
[db-sync-node:Info:6] [2021-09-21 22:15:06.39 UTC] localInitiatorNetworkApplication: connecting to node via "../cardano-node/state-node-mainnet/node.socket"
[db-sync-node.Subscription:Notice:71] [2021-09-21 22:15:06.39 UTC] Identity Starting Subscription Worker, valency 1
[db-sync-node.Subscription:Notice:72] [2021-09-21 22:15:06.39 UTC] Identity Connection Attempt Start, destination LocalAddress "../cardano-node/state-node-mainnet/node.socket"
[db-sync-node.Subscription:Notice:72] [2021-09-21 22:15:06.39 UTC] Identity Connection Attempt End, destination LocalAddress "../cardano-node/state-node-mainnet/node.socket" outcome: ConnectSuccessLast
[db-sync-node.Handshake:Info:72] [2021-09-21 22:15:06.39 UTC] WithMuxBearer (ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"}) Send (ClientAgency TokPropose,MsgProposeVersions (fromList [(NodeToClientV_1,TInt 764824073),(NodeToClientV_2,TInt 764824073),(NodeToClientV_3,TInt 764824073),(NodeToClientV_4,TInt 764824073),(NodeToClientV_5,TInt 764824073),(NodeToClientV_6,TInt 764824073),(NodeToClientV_7,TInt 764824073),(NodeToClientV_8,TInt 764824073),(NodeToClientV_9,TInt 764824073)]))
[db-sync-node.Handshake:Info:72] [2021-09-21 22:15:06.39 UTC] WithMuxBearer (ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"}) Recv (ServerAgency TokConfirm,MsgAcceptVersion NodeToClientV_9 (TInt 764824073))
[db-sync-node.Mux:Info:72] [2021-09-21 22:15:06.39 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: Handshake Client end, duration 0.000210857s
[db-sync-node.Mux:Info:73] [2021-09-21 22:15:06.39 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: State: Mature
[db-sync-node:Info:76] [2021-09-21 22:15:06.39 UTC] Starting chainSyncClient
[db-sync-node:Info:76] [2021-09-21 22:15:06.40 UTC] Cardano.Db is empty
[db-sync-node:Info:81] [2021-09-21 22:15:06.40 UTC] Running DB thread
[db-sync-node.Subscription:Notice:71] [2021-09-21 22:15:06.42 UTC] Identity Required subscriptions started
[db-sync-node:Info:81] [2021-09-21 22:15:06.42 UTC] Rolling back to genesis
[db-sync-node:Info:81] [2021-09-21 22:15:06.42 UTC] No blocks need to be deleted
[db-sync-node:Info:81] [2021-09-21 22:15:06.43 UTC] Found snapshot file for genesis
[db-sync-node:Info:84] [2021-09-21 22:15:06.55 UTC] getHistoryInterpreter: acquired
[db-sync-node:Info:96] [2021-09-21 22:15:06.55 UTC] Running Offline fetch thread
[db-sync-node:Info:81] [2021-09-21 22:15:06.55 UTC] thisIsAnUglyHack: Main thead
[db-sync-node:Info:81] [2021-09-21 22:15:06.55 UTC] Starting at epoch 0
[db-sync-node:Error:81] [2021-09-21 22:15:06.56 UTC] Error: insertABOBBoundaryblock hash 47a50064150accb15f33e96d1c3ecac988fbd3d682e45fc54d6f2d89857f05b7
[db-sync-node:Info:81] [2021-09-21 22:15:06.56 UTC] Shutting down DB thread
[db-sync-node.Mux:Notice:73] [2021-09-21 22:15:06.56 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: Miniprotocol (MiniProtocolNum 5) InitiatorDir terminated cleanly
[db-sync-node.Subscription:Notice:71] [2021-09-21 22:15:07.56 UTC] Identity Restarting Subscription after 1.16461565s desired valency 1 current valency 0
[db-sync-node.Subscription:Notice:71] [2021-09-21 22:15:16.40 UTC] Identity Starting Subscription Worker, valency 1
[db-sync-node.Subscription:Notice:105] [2021-09-21 22:15:16.40 UTC] Identity Connection Attempt Start, destination LocalAddress "../cardano-node/state-node-mainnet/node.socket"
[db-sync-node.Subscription:Notice:105] [2021-09-21 22:15:16.40 UTC] Identity Connection Attempt End, destination LocalAddress "../cardano-node/state-node-mainnet/node.socket" outcome: ConnectSuccessLast
[db-sync-node.Handshake:Info:105] [2021-09-21 22:15:16.40 UTC] WithMuxBearer (ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"}) Send (ClientAgency TokPropose,MsgProposeVersions (fromList [(NodeToClientV_1,TInt 764824073),(NodeToClientV_2,TInt 764824073),(NodeToClientV_3,TInt 764824073),(NodeToClientV_4,TInt 764824073),(NodeToClientV_5,TInt 764824073),(NodeToClientV_6,TInt 764824073),(NodeToClientV_7,TInt 764824073),(NodeToClientV_8,TInt 764824073),(NodeToClientV_9,TInt 764824073)]))
[db-sync-node.Handshake:Info:105] [2021-09-21 22:15:16.40 UTC] WithMuxBearer (ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"}) Recv (ServerAgency TokConfirm,MsgAcceptVersion NodeToClientV_9 (TInt 764824073))
[db-sync-node.Mux:Info:105] [2021-09-21 22:15:16.40 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: Handshake Client end, duration 0.000231694s
[db-sync-node.Mux:Info:106] [2021-09-21 22:15:16.40 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: State: Mature
[db-sync-node:Info:109] [2021-09-21 22:15:16.40 UTC] Starting chainSyncClient
[db-sync-node:Info:109] [2021-09-21 22:15:16.41 UTC] Cardano.Db is empty
[db-sync-node:Info:114] [2021-09-21 22:15:16.41 UTC] Running DB thread
[db-sync-node:Info:114] [2021-09-21 22:15:16.41 UTC] Rolling back to genesis
[db-sync-node:Info:114] [2021-09-21 22:15:16.41 UTC] No blocks need to be deleted
[db-sync-node:Info:114] [2021-09-21 22:15:16.41 UTC] Found in memory ledger snapshot at genesis
[db-sync-node.Subscription:Notice:71] [2021-09-21 22:15:16.43 UTC] Identity Required subscriptions started
[db-sync-node:Error:114] [2021-09-21 22:15:16.43 UTC] Error: insertABOBBoundaryblock hash 47a50064150accb15f33e96d1c3ecac988fbd3d682e45fc54d6f2d89857f05b7
[db-sync-node:Info:114] [2021-09-21 22:15:16.43 UTC] Shutting down DB thread
[db-sync-node.Mux:Notice:106] [2021-09-21 22:15:16.43 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: Miniprotocol (MiniProtocolNum 5) InitiatorDir terminated cleanly

And here is what shows up on the terminal of the node:

[BB-Ubunt:cardano.node.LocalErrorPolicy:Error:94] [2021-09-21 22:15:07.56 UTC] IP LocalAddress “” ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed “<socket: 50> closed when reading data, waiting on next header True”)

[BB-Ubunt:cardano.node.LocalErrorPolicy:Error:94] [2021-09-21 22:15:16.43 UTC] IP LocalAddress “” ErrorPolicyUnhandledApplicationException (MuxError (MuxIOException writev: resource vanished (Broken pipe)) “(sendAll errored)”)

[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:14:35.93 UTC] Chain extended, new tip: 33106138ff06e609aa3689d4a99648e299989d7f609dad03db50b4e9edafdaf9 at slot 40696184
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:14:40.45 UTC] Chain extended, new tip: 50d2891feaafecce4b3bb039aa608f2adc7c10ec8ed39952577d70d85eb13927 at slot 40696189
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:14:42.36 UTC] Chain extended, new tip: 5e032a82352a614d756b1a09d9fa621419dd903bace1bc111ba24bd61b0fe1c8 at slot 40696191
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:15:03.53 UTC] Chain extended, new tip: 0a84a79478f7de6c121e0e628567e80400f6c1461097ce2a7fc9101cf8a88786 at slot 40696212
[BB-Ubunt:cardano.node.LocalErrorPolicy:Error:94] [2021-09-21 22:15:07.56 UTC] IP LocalAddress "" ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed "<socket: 50> closed when reading data, waiting on next header True")
[BB-Ubunt:cardano.node.LocalErrorPolicy:Error:94] [2021-09-21 22:15:16.43 UTC] IP LocalAddress "" ErrorPolicyUnhandledApplicationException (MuxError (MuxIOException writev: resource vanished (Broken pipe)) "(sendAll errored)")
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:15:22.62 UTC] Chain extended, new tip: fb4d6277574e0b8a2ad41cef6aa31326c2520ac1bfa8e9461c1d8aa1eba01b47 at slot 40696231
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:16:00.59 UTC] Chain extended, new tip: 64afeda82f255cfa93835ef0a559f425b5f5de02d922c70cbda7f05c4d4bd998 at slot 40696269
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:16:48.03 UTC] Chain extended, new tip: 6524197c55fbd148402633e05978c264d461d9bcfdd2ab443bb09d6eee2f967c at slot 40696316
[BB-Ubunt:cardano.node.ChainDB:Notice:63] [2021-09-21 22:16:58.39 UTC] Chain extended, new tip: 3dc680d6b0b90f79565d1814eb8a264358791b8b10c6a4a2157bf8163fc6e68b at slot 40696327

This is on a fresh install of Ubuntu 20.04.3 LTS. Any help is appreciated.

4

That error only occurs on the master branch (and has since been fixed).

I (as the main dev on db-sync) highly recommend that people do not use the master branch because it does occasionally break (this is an example). Instead people should be using the latest official release/tag.

5

Hi Erik,

Thanks for all the effort.
I ran into the same issue/error, thanks for the release branch hint, after switching to the latest release branch (git clone -b release/11.0.x …), all errors disappeared.

Innar

6

Thanks Erik. Switching the branch fixed that error.

I am now getting a different error when running cardano-db-sync:

[db-sync-node:Error:77] [2021-09-26 12:18:34.23 UTC] runDBThread: DbInsertException "EpochParam" (SqlError {sqlState = "42703", sqlExecStatus = FatalError, sqlErrorMsg = "column \"cost_models\" of relation \"epoch_param\" does not exist", sqlErrorDetail = "", sqlErrorHint = ""})
[db-sync-node:Error:80] [2021-09-26 12:18:34.23 UTC] recvMsgRollForward: AsyncCancelled
[db-sync-node:Error:72] [2021-09-26 12:18:34.23 UTC] ChainSyncWithBlocksPtcl: DbInsertException "EpochParam" (SqlError {sqlState = "42703", sqlExecStatus = FatalError, sqlErrorMsg = "column \"cost_models\" of relation \"epoch_param\" does not exist", sqlErrorDetail = "", sqlErrorHint = ""})
[db-sync-node.Mux:Info:69] [2021-09-26 12:18:34.23 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: State: Dead
[db-sync-node.Mux:Notice:69] [2021-09-26 12:18:34.23 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: Miniprotocol MiniProtocolNum 5 InitiatorDir terminated with exception DbInsertException "EpochParam" (SqlError {sqlState = "42703", sqlExecStatus = FatalError, sqlErrorMsg = "column \"cost_models\" of relation \"epoch_param\" does not exist", sqlErrorDetail = "", sqlErrorHint = ""})
[db-sync-node.Subscription:Error:68] [2021-09-26 12:18:34.23 UTC] Identity Application Exception: LocalAddress "/home/brandon/state-node-mainnet/node.socket" DbInsertException "EpochParam" (SqlError {sqlState = "42703", sqlExecStatus = FatalError, sqlErrorMsg = "column \"cost_models\" of relation \"epoch_param\" does not exist", sqlErrorDetail = "", sqlErrorHint = ""})
[db-sync-node.ErrorPolicy:Error:6] [2021-09-26 12:18:34.23 UTC] IP LocalAddress "/home/brandon/state-node-mainnet/node.socket" ErrorPolicyUnhandledApplicationException (DbInsertException "EpochParam" (SqlError {sqlState = "42703", sqlExecStatus = FatalError, sqlErrorMsg = "column \"cost_models\" of relation \"epoch_param\" does not exist", sqlErrorDetail = "", sqlErrorHint = ""}))

Is there a problem with my schema?

7

The versions of the schema on the master branch is different from the schema on the release branch.

You will need to drop the database, recreate (as an empty database) and next time you run db-sync the database will be updated as needed.

1 Like
8

Hi, @erikd what branch would you recommend apart from master ?

9

I always recommend the latest release tag. Currently its tag 12.0.1