Shelley Incentivized Testnet Development Update - 14 February 2020

We’ve rolled two weeks’ development into one release - here’s what to expect from IOHK Project Manager, Dimitris Poulopoulos:

For those of you who like to read, here’s a script of the video :slight_smile:

Hi everyone. It is Dimitris again, back for another week to share a quick update on the Shelley Incentivized Tesntet progress. Thanks for all your further feedback on these weekly video updates. It’s been great to get your thoughts on how we can improve them. We’re glad you are finding them useful and we will continue to try to bring you more useful information going forward.


Network stability still remains our focus and we have seen significant node availability improvements since we pushed out 0.8.9 with uptime increasing to well over 90% average. Last week, we decided to forego the regular Wednesday update of Jormungandr on the Incentivized Testnet in order to target an end to end release for this week and make available some new features too.

This week

So that brings us to this week. We have now released a new Node version 0.8.10-2. At the time I am recording this video we are finalising the integration with Daedalus and Wallet BE. We hope to make those available later today or shortly after the weekend depending on QA activities progress. Together, we hope this will continue to improve the experience for both stake pool operators and ADA holders.


This week’s Jormungandr release contains two weeks worth of work which should bring improvements in chain selection and quality and it should help with forking issues. The highlights of the release include improvements in the asynchronous/multithreading management, and better handling of the client connections by the node.

Updates to asynchronous code
The team has been eager to start porting one year of future 0.1 work to the async/.await stack. This has been started and allowed us to clean up some of the performance issues and locking issues we had.

The leadership, the chain, the REST and the fragment pool modules of the node have been updated and improved. There are now less blocking points risking to create delay in the leader events (stake pool operator will appreciate that). Management of client-side network connections

There is now a limit on client-side gRPC connections kept alive to receive subscription updates from peers, governed by configuration parameter max_client_connections. The intent is to reduce the overall number of idle connections in the network and improve connection reliability for non-public nodes behind NAT.

Add peer queries to the network protocol. This will be used in a future update to take the bootstrapping load off trusted peers. Integrate lock-free Multiverse in mockchain. Remove synchronous locking from the so-called “multiverse” data structure. This is part of the work to remove the observed performance problems.

Improve logging of the stuck notifier
Impose a separate smaller limit on persistent client-side connections, defaulting to 8 connections at a time. This was done by limiting the number of client gRPC connections, use GC to close excess. This should reduce the number of idle connections in the network, and make connectivity more robust for non-public nodes behind a NAT.

Bug fixes
Report the stake for the selected pool instead of the whole system stake Observed error: too late, system time might be off? Which fixes an annoying lockup bug that was first reported in 0.8.6 version.

Input process async. Porting parts of jormungandr to std::future (the modern Rust async API).
Do not truncate the list of peers for propagation; to revert a change done in the previous version that negatively affected network homogeneity.
Update ledger ed25519 signing to the new signing closure

More details may be found as always on the Jormungandr repo under the latest release tag

Daedalus and Wallet BE

The new features we will be introducing in Daedalus along with changes in the Wallet BE are aimed to improve syncing performance and introduce new capabilities such as Yoroi support and HW Wallet restoration. The team has also added Stake Pool ranking an experimental feature. This will need some further tuning over the weeks ahead but please do check it. We have also made some backend changes which should help in resolving the zero balance occurrences still being seen by some of the users.

Some of the key items include:
Ledger and Trezor hardware wallet restoration
Yoroi wallet support
Resyncing wallet with the blockchain
Stake pool ranking based on desirability
Stake pool details now include saturation
Zero issue balance fix

We will let you know as soon as this new Wallet version is available.

Coming next week

Now that the network is more stable than before, we have finally started the move to Rust’s async/await. This is a really exciting time for the team as it was a long awaited feature of the Rust Programming Language. This will allow us to write cleaner and more concise code and also to improve performance a lot.

Network Blacklisting from config file
Spikes of TCP connections before getting stuck
Enclave avoid duplicated Leaders

Side updates

We also have a few community-generated content pieces this week!

Support: Syncing and connection related tickets still comprise the majority of tickets received by our support team. As we continue to enhance the network there should be a reduction in these incidents. For users who are awaiting a fix, tickets are placed in on-hold status, contributing to our backlog. If you are waiting on a resolution to a syncing or connection related issue, you can expect an update from the IOHK Technical Support Desk in the coming days.

New Incidents - 671
Resolved Incidents - 513
Incident Backlog - 1,314

DevOps: We’ve increased the amount of connections allowed from peers without reachable IP addresses (e.g. Daedalus clients).

I hope you found this short update useful.

I’ll be back next week with the latest on ITN progress. Thank you for watching