Marlowe resources and tools (playground, tutorials, documentation)

Developers Smart Contracts
42
43

Thanks for helping out with various options.

However so far, I was using the docker compose from below repo as suggested and tested everything end to end.

marlowe-starter-kit/docker-compose.yaml at main · input-output-hk/marlowe-starter-kit · GitHub

But today I noticed there is no Mainnet folder here (just preprod and preview), hence I dig further and found your PR for mainnet raised 2 days ago.
input-output-hk/marlowe-starter-kit at PLT-6352 (github.com)

Than I cloned this branch and try to use the docker compose but its not at all working.
I even went further and found some missing files for mainnet and created them manually from here:

marlowe-starter-kit/preprod/node at main · input-output-hk/marlowe-starter-kit · GitHub

Still marlowe-starter-kit_indexer_1 is not coming up and see lot of errors in docker:

tx_1 | 2023-06-24T17:52:45.492895954Z [Error] [Control.Concurrent.Component.component#90] Component tx-job-server-worker crashed with exception:
tx_1 | 2023-06-24T17:52:45.493060157Z [Error] [Control.Concurrent.Component.component#91] Data.Binary.Get.runGet at position 0: not enough bytes

I never thought switching to mainnet will be that hard.

Please advise.

Thanks,
AS

44

Still marlowe-starter-kit_indexer_1 is not coming up and see lot of errors in docker:

tx_1 | 2023-06-24T17:52:45.492895954Z [Error] [Control.Concurrent.Component.component#90] Component tx-job-server-worker crashed with exception:
tx_1 | 2023-06-24T17:52:45.493060157Z [Error] [Control.Concurrent.Component.component#91] Data.Binary.Get.runGet at position 0: not enough bytes

This is a known bug where marlowe-tx and marlowe-sync crash while the Cardano node is still in the Byron era. Even though it’s annoying, it is harmless and will go away when the node reaches the Shelley era.

Depending upon the hardware, it may take hours or days for the node and Marlowe runtime to sync to the tip of the blockchain. The log for marlowe-starter-kit_node_1 will tell you what slot number the node is synced to. A query like the following will show what slot the databases for marlowe-chain-indexer and marlowe-indexer are on:

docker exec -it -u postgres marlowe-starter-kit_postgres_1 psql -d chain_mainnet -c 'select max(blockno) as blockno, max(slotno) as slotno from chain.block union all select max(blockno), max(slotno) from marlowe.block'

There are unofficial database snapshots available at http://data.marlowestat.org/snapshots/, but there aren’t instructions for using them.

Demeter run already has mainnet synced for Node and Runtime: Cardano Marlowe Runtime. There is a grant program that can provide free computing for Marlowe Runtime.

45

Hello @bwbush ,

Could you please confirm that running marlowe-runtime and cardano-cli on mainnet (preprod it works all good) is possible or not.
We dont have time to try new stuff, I am aware of other available solutions but we are very close to our live date and dont have time to try new things.

If its possible, could you please guide me to the correct repo/image to use mainnet, I never thought switching to mainnet will be that painful.
I saw your PR for mainnet and created the missing file in my local cloned repo for marlowe starter kit.

But its not working, can you please help or else we have no choice but ditch cardano smart contract.

Thanks once again for your help.

Thanks,
AS

46

Also to give you more context, I can see chain.block are getting synced as I can see it progressing but marlowe.block has no data and marlowe-runtime commads are not working.

I suspect some issue with postgress DB and its not properly set up. I am ready to drop everything and recreate but which repo to use.

Please advice @bwbush

47

Yes, I used Marlowe Runtime to execute a contract on mainnet yesterday: 56b1f3392793f836a5d58c25c90a71e43c43c691155c8caa1623b15f28a79006#1

The PLT-6352 branch of the starter kit enables mainnet. Moreover, if you use the Marlowe Extension for Demeter Run, then you don’t need to install anything or wait for the node and database to sync.

The marlowe.block table in the database will remain empty until block 7791724 (= slot 72317305) is reached because that is the first Marlowe transaction on mainnet that used the audited validator.

The last time I sync’ed mainnet on my development machine, it took about 16 hours. However, syncing is I/O limited by the PostgreSQL database, so it depends a lot upon hardware and on whether the PostgreSQL database is tuned for that particular hardware. For example, syncing from mainnet genesis on a GCE E2 (Intel Broadwell) machine without any PostgreSQL tuning takes about 6 days.

Feel free to message me directly if you’d like temporary access to an already sync’ed mainnet deployment of Marlowe Runtime.

48

Thanks a lot @bwbush .

I am trying to set up everything fresh on new machine, referrred your PR and created the missing init.sql file.
One question inside preprod and preview folder there is a node folder on main branch which is not there in your PR. Do I have to create that manually or not.

Thanks,
AS

49

I don’t think you need to create the node/ folder manually, but I’m not 100% sure. It’s harmless to create it manually, however.

50

Thanks for quick reply @bwbush

What about the file inside node folder like alonzo-genesis.jon, config.json, shelley-genesis.json, byron-genesis.json etc
?

Shall i replicate the files from preprod
?

-AS

51

So sorry, of course those should be in the node folder. I just updated the branch GitHub - input-output-hk/marlowe-starter-kit at PLT-6352 to include them.

52

Ohhh, I already started docker-compose ps -d , I hope creating those files later does cause any harm to the set up.

If yes, let me know how to un do that please.

-AS

53

Stopping and then starting the node should be sufficient. Then its log file should show that it is syncing after its startup is complete.

54

What about the remaining files
?
I found this list:
alonzo-genesis.json
byron-genesis.json
config.json
conway-genesis.json
db-sync-config.json
shelley-genesis.json
submit-api-config.json
topology-p2p.json
topology.json

Are they not needed or auto created @bwbush

-AS

55

Only the four files are needed (config.json, byron-genesis.json, shelley-genesis.json, alonzo-genesis.json) are needed. Restarting the docker containers should be sufficient, docker-compose restart.

56

I really appreciate your timely help @bwbush .

Everything came up post restart except below which is in restarting mode. I hope this will come up soon.

marlowe-starter-kit_indexer_1 /bin/entrypoint Restarting

Also as expected chain.block started syncing but there is nothing yet in marlowe.chain, which is expected as explained by you above it will start later post block 7791724 .

I see below errors in docker as of now, I hope they are fine and will of off eventually.:

[Error] [Control.Concurrent.Component.component#90] Component contract-load-worker crashed with exception:
[Error] [Control.Concurrent.Component.component#91] Data.Binary.Get.runGet at position 0: not enough bytes
CallStack (from HasCallStack):

[Error] [Control.Concurrent.Component.component#90] Component marlowe-runtime-worker-traced crashed with exception:
[Error] [Control.Concurrent.Component.component#91] Data.Binary.Get.runGet at position 0: not enough bytes
CallStack (from HasCallStack):
error, called at src/Data/Binary/Get.hs:351:5 in binary-0.8.8.0-HpmhsQDJ8BC4iODE4wIoDZ:Data.Binary.Get
[Error] [Control.Concurrent.Component.component#90] Component sync-query-worker crashed with exception:

nc: connect to proxy (172.19.0.10) port 3701 (tcp) failed: Connection refused
Connecting to proxy:3701 failed, sleeping for 1 seconds
nc: connect to proxy (172.19.0.10) port 3701 (tcp) failed: Connection refused
Connecting to proxy:3701 failed, sleeping for 2 seconds
nc: connect to proxy (172.19.0.10) port 3701 (tcp) failed: Connection refused
Connecting to proxy:3701 failed, sleeping for 4 seconds
nc: connect to proxy (172.19.0.10) port 3701 (tcp) failed: Connection refused
Connecting to proxy:3701 failed, sleeping for 8 seconds
Connection to proxy (172.19.0.10) 3701 port [tcp/*] succeeded!

[Control.Concurrent.Component.component#90] Component chain-seek-worker crashed with exception:

Please check and suggest if something need to be done for any of them.

Thanks a ton for your support, I hope I will be able to complete this by tomorrow EOD and get something to demo.

Thanks,
AS

57

These messages should go away once the node reaches the Shelley era. In the Byron era, the marlowe-chain-indexer logs should not show any errors, but in the Shelley era none of the services should show errors. (You can use docker logs --timestamps --tail 50 . . . to see when the errors occurred and just view the last few lines.)

Whether this syncs before EOD tomorrow will depend upon the speed of the disk storage, since syncing is i/o bound.

58

Hello @bwbush, thanks for helping out yesterday.

After overnight sync I can see chain.block is synced to 19222867 which is higher to 7791724. So i expect marlowe.block should start getting synced as well but I still see nothing there.

And, I see lot of below errors, let me know if this is expected and will go away with sync.

2023-06-27T09:29:24.766035058Z [55831a41:cardano.node.LocalErrorPolicy:Error:69] [2023-06-27 09:29:24.76 UTC] IP LocalAddress “/ipc/node.socket@20” ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed “<socket: 29> closed when reading data, waiting on next header True”)

2023-06-27T09:29:34.995488676Z [55831a41:cardano.node.LocalErrorPolicy:Error:69] [2023-06-27 09:29:34.99 UTC] IP LocalAddress “/ipc/node.socket@21” ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed “<socket: 27> closed when reading data, waiting on next header True”)

2023-06-27T09:29:45.173616500Z [55831a41:cardano.node.LocalErrorPolicy:Error:69] [2023-06-27 09:29:45.16 UTC] IP LocalAddress “/ipc/node.socket@22” ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed “<socket: 29> closed when reading data, waiting on next header True”)

2023-06-27T09:29:55.788634393Z [55831a41:cardano.node.LocalErrorPolicy:Error:69] [2023-06-27 09:29:55.78 UTC] IP LocalAddress “/ipc/node.socket@23” ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed “<socket: 27> closed when reading data, waiting on next header True”)

proxy_1 | 2023-06-27T09:26:42.435046978Z [Error] [Control.Concurrent.Component.component#90] Component marlowe-runtime-worker-traced crashed with exception:
proxy_1 | 2023-06-27T09:26:42.435191679Z [Error] [Control.Concurrent.Component.component#91] Data.Binary.Get.runGet at position 0: not enough bytes
proxy_1 | 2023-06-27T09:26:42.435238280Z CallStack (from HasCallStack):
proxy_1 | 2023-06-27T09:26:42.435275680Z error, called at src/Data/Binary/Get.hs:351:5 in binary-0.8.8.0-HpmhsQDJ8BC4iODE4wIoDZ:Data.Binary.Get
sync_1 | 2023-06-27T09:26:41.546952196Z [Error] [Control.Concurrent.Component.component#90] Component sync-query-worker crashed with exception:
sync_1 | 2023-06-27T09:26:41.546987696Z [Error] [Control.Concurrent.Component.component#91] Data.Binary.Get.runGet at position 0: not enough bytes
sync_1 | 2023-06-27T09:26:41.546994096Z CallStack (from HasCallStack):
sync_1 | 2023-06-27T09:26:41.546998297Z error, called at src/Data/Binary/Get.hs:351:5 in binary-0.8.8.0-HpmhsQDJ8BC4iODE4wIoDZ:Data.Binary.Get
tx_1 | 2023-06-27T09:26:40.557837032Z Connection to chain-sync (172.18.0.5) 3715 port [tcp/*] succeeded!

Thanks,
AS

59

Also below service is still in restarting state:
marlowe-starter-kit_indexer_1

And when I try to query tip I get below:
as@clvmas:~/marlowe-starter-kit$ cardano-cli query tip --mainnet
cardano-cli: Network.Socket.connect: <socket: 11>: permission denied (Permission denied)

And when I try to query the contract ID you shared above, I get contract not found:
marlowe-runtime-cli log “56b1f3392793f836a5d58c25c90a71e43c43c691155c8caa1623b15f28a79006#1”

I suspected something wrong with node socket path, but i can see its correctly set as shown below

as@clvmas:~/marlowe-starter-kit$ echo $CARDANO_NODE_SOCKET_PATH
/var/lib/docker/volumes/marlowe-starter-kit_shared/_data/node.socket

Any of the commands are not working, only thing working is curl -sSI “http://127.0.0.1:3780/healthcheck”

below is the status of blocks:
~/marlowe-starter-kit$ docker exec -it marlowe-starter-kit_postgres_1 psql -U postgres chain_mainnet -c ‘select max(slotno) from chain.block;’
max

19222867 (This number is not changing since morning, I suspect this is synced completely.)
(1 row)

~/marlowe-starter-kit$ docker exec -it marlowe-starter-kit_postgres_1 psql -U postgres chain_mainnet -c ‘select max(slotno) from marlowe.block;’
max

                   (Here nothing so far)

(1 row)

60

@bwbush : Can you help
?

61

Did you set permissions on the node socket: sudo chmod a+rwx $CARDANO_NODE_SOCKET_PATH?

The indexer should not be crashing after the Byron era has passed.

I suggest you open a help ticket so that a support engineer can assist with debugging this.