Shelley Incentivized Testnet Development Update - 31January 2020

Developers Misc Dev Talk
1

In this Shelley Incentivized Testnet Development Update video series, Project Manager Dimitris Poulopoulos will be taking you through the updates on a regular basis:

For those of you who like to read, here’s a script of the video :slight_smile:

Hi everyone. It is Dimitris again, project manager for the Shelley Incentivized testnet. Back again for another week to share a quick update on ITN progress.

Explanation

Thanks for your further feedback on these weekly video updates, it’s been great to see the continued level of engagement.

We will keep providing these updates weekly to keep you up to speed with all the work we’re doing, all the learnings and the improvements we’re making to the testnet.

Recap
Network stability remains our number one priority. We’ve already seen some further minor improvements in the Testnet since we pushed out 0.8.7 last week and we are continuing to work towards this direction.

This week’s highlights
As you might have noticed already there have been 2 releases out this week, 0.8.8 and 0.8.9 instead of just the one expected. The latter was made available to address bootstrapping issues which have been observed during e2e QA testing of 0.8.8. So please install directly 0.8.9 that includes the latest regression fix too.

0.8.8 aims to provide further minor improvements to logging, bootstrapping and some temporary relief fixes for CPU/locking pains. More specifically:

  1. Improvements to logging in storage bootstrap, network tasks and chain-selection. This is expected to provide more diagnostic information on the issues seen with the node deployment.
  2. Improve bootstrapping reliability by retrying, expected to improve the connection success.
  3. Reduce the CPU impact of poldercast algorithm with a temporary fix; in an effort to improve performance and reduce observed congestion issues.
  4. Reduce the issues of synchronous locking on the asynchronous stack while we carry on with the migration to asynchronous. Expected to reduce the possibility of interthead lockups previously observed. This is ongoing and upcoming releases will include further fixes.

0.8.9 was released shortly after (the next day) to fix panics observed during network bootstrap where a block was ahead of the validated ones (not in sync with the main chain) as well as introducing the use of checkpoint when bootstrapping. With the latter being a feature improvement to address too many blocks being sent in case the previous blockchain state was in a fork that was not the main chain.

  • Further enhancements:

    • Increase selection information in the existing chain-selection/application #1670 to improve available logging.
    • Bootstrap with large number of blocks looks blocked #1669, to inform users of the progress of bootstrapping process via improved logging.
    • Update chain-deps #1674 to bring the latest fixes and improvements from the chain-libs libraries.
    • Fixes bootstrapping from trusted peers falling through when node is n… #1646 to add a mode suitable for trusted peer operation where is it not expected to bootstrap and changes default behaviour to retry bootstrapping upon failure.

  • Bug fixes

    • DatabaseLocked error when using in-memory database #1638 to remove the occurence of locking failures due to DB concurrence. Which was addressed by adding an Integrate fix for DatabaseLocked error #1641.

  • Some of the Issues addressed (indicative selection)

    • As a small stake pool operator (and hopefully as “anyone else”), I would like to have incentives baked into a Cardano Constitution (and the Ouroboros protocol), so that we can assure the ongoing decentralization of Cardano ecosystem. #1657 - closed by user following further clarifications provided by the devs.
    • Competitive fork slot and timestamps do not match #1651, closed by reporter without further analysis.
    • Compiled v0.8.7 jcli binary doesn’t execute MacOS Catalina #1644 closed by reporter without further analysis.
    • Tokio-runtime PoisonError Panic, v0.8.8 #1643 that was fixed by the changes introduced in 0.8.7.
    • Rest API stops responding while Jormungandr continues to run #1642 which was closed as a duplicate, still looking at the initial issue.
    • Remote node not yet fully connected should not be picked for fetching block, otherwise it fails - block fetch from xxxx failed: PropagateError { kind: NotSubscribed,… #1630 as addressed from 0.8.7.
    • Different leader schedule showing on 0.8.7 than 0.8.6 #1624

  • Merged pull requests:

    • Circa 20 requests have also been merged. Please see the detailed change log for more details.

  • More release details available on Github under the Jormungandr repo https://github.com/input-output-hk/jormungandr/releases/tag/v0.8.8 and https://github.com/input-output-hk/jormungandr/releases/tag/v0.8.9

Coming soon

  • There is still road ahead for improving Jormungandr stability further and some of the items planned for the next release include:

    • addressing errors related to system time
    • Implementing a limit at the number of client gRPC connections, to close off excessive connections that have not been used for a while. To help avoid having an explosion of connections in aggregate which resulted in saturating some nodes.
    • Network Blacklisting from config file

  • Further enhancements that the rest of the teams are also working on will be released during the coming weeks in conjunction with confirming further network stability improvements in order to make sure enhanced user experience is not affected by the current network stability issues.

If you found this update useful please let us know.

Thank you for watching!

4 Likes