Everything that's wrong with Catalyst (now with poll)

Governance Project Catalyst
28

So if I’m reading this correctly, you just want one nice website that has something like “everything you need to understand, improve and discuss Catalyst” right?

I’m all for it. One of the things I raised was precisely the creation of such a gathered place. This is why Ideascale now has these types of links (except the “how to change catalyst” because it isn’t defined, and the reason it’s not defined is because I think IOG wants that process to emerge from within the community/they are still developing the model, see the treasury white paper)

No. UX sucks (as it does generally in web3 tbh). I saw a problem and, since I decided to also be a PA when I submitted a proposal, I learned about how it worked so I joined the PA chat. After seeing so many flaws I just couldn’t let it pass without at least trying to fix it…

There was a “Issues Dropbox” created by IOG (Daniel R) but even that was super obscure to find.

When we created this feedback document “TODO” for fund 10, then we simply asked IOG community Shepard Daniel to include it the weekly newsletter. I wish there was a more regularly maintained and “nice” website (PACE that hosts projectcatalyst.org would be a good candidate, but he’s just one guy and there’s a lot)…

I agree! That’s why one of the top priority tasks is to create a process, with the community. Do you want to pick up a shovel and join? Or do you want to hand it over to IOG and hope for the best?

Yup. And I think they have been expanding their team… but if I’m gonna guess, they wanted to community to self-finance these efforts through project catalyst. Which we’ve partially done, but there just aren’t enough people who care enough and have been willing to put in time + the fact that “governance/community” type challenges weren’t funded in the PAST TWO FUNDS.

It’s not official but as close as it gets. All people who have been involved in all the changes and guidelines writing from the past funds are there. There’s also a treasury financing efforts. It is the most promising venue, assuming you do put in the work needed. There is this endeavour which seeks to gather all initiatives and ideas to this one decentralised task management platform!

There’s no other competing initiative. AIM, PACE, SWARM , CC admin, community governance and oversight, and Catalyst school are other community initiatives but they’re not primarily about discussing, improving, and changing the system.

Nothing I disagree with here. I’m just giving you a path forward! Seeing how this has been the case for so long, we’re a few people who are taking matters into our own hands. You can choose to join in and increase our chances of getting changes through by fund 10 or you can keep on 0screaming into the void” as you aptly put it!

How exactly is it made “inaccessible by the in-group nature of the space” in your view (I can make a guess but I don’t want to put words in your mouth)? Is it the PA telegram group, the town halls, the difficulty in finding where everything is/how to change things?

What should be done (eg what could SWARM do) to make things better?

Just trying to take input so that more people can get involved! I’m not part of the community groups whose purpose is to build community and educate but I could forward things and create bounty tasks etc

1 Like
29

Would be a first step.

In the end, it should also host the things in my “Results are Invisible” and “Technology is Intransparent Patchwork Rug” points from the original post.

Replacing Ideascale might maybe take longer, but hosting the voting results there – with more details and CSV versions for people who want to do analyses on them – should be doable in days.

The voting dApp and the presentation of the previously funded projects and their results – which was already promised – have to live somewhere, anyway. So, they should, of course, live on that site instead of being scattered somewhere on the Internet.

Personally, I’d love a very structured approach, where everything – past and current challenges, categories(?), proposals, assessments, voting results, … – is not only available as human-readable web pages, but also in machine-readable form, so that wallets, voting apps, analyses and evaluations have easy access.

I know that ideas are easier said than done in IT, I have enough construction sites at my (very much not crypto-related) work, but I also know that at least a start – setting up a small static website or an effing Wordpress and put the most important documents and links on it – is doable quickly. And I really have no understanding that it was not done at the very beginning.

Both!

I don’t particularly like the “pick up a shovel” response. The funds of the treasury somehow belong to all ADA holders, not just the ones who have the time, energy, and ideas to “join”. It is totally legitimate to want to raise an opinion on how they are managed without having the resources to go into it in depth. It is even legitimate to conclude that it’s better if they are not distributed at all than like that, that they should better just go into staking rewards. “Come into our time-draining space and do something!” is perfect derailing in that case.

But I do like my proposals enough that I’m inclined to invest that time.

Some of them do need something from IOG. For example, the “Combinable Public Recommendations” could be prototyped in a third-party voting application, but for that such applications would have to be made possible. I’d need an API, to not only get the signatures from the voters as in CIP 62, but also to submit the votes to their intransparent backend.

(Some technological deep dives would also be a welcome content for said central, authoritative Catalyst hub website. I found no other way to get to know what happens between the voting app and the Google Docs than ask some people who are here years longer than me. That’s not really worthy of a cryptocurrency – “Don’t trust! Verify!” – solution.)

More importantly, there needs to be a commitment by IOG to not just take some alleged consensus from the highly engaged in-group, but take the discussion and decision to the wider Cardano community, which – it seems – does have opinions on Catalyst that might not be aligned with the ones of the very active people on the inside.

In fact, we see that sometimes in voting results, when some proposals that were so well-received in the Catalyst bubble do not get funding, because the general Cardano public thinks differently, did not get the information, the Catalyst bubble does not see which impression you get if you mainly see it as a headline with some stars in the voting app.

…, which might very well have been the case here.

Didn’t grasp it on first sight. Seems to be something to set tasks and promise payments for people, not necessarily make visible competing solutions to the same problems and decide between them. But will look again and ask there. … But I’m quite sure, it’s not decentralised at all. It’s one totally centralised, totally Web 2.0 website – dework.xyz – linked to another totally centralised, totally Web 2.0 website – discord.com.

Yes, basically: Where and what is everything? Who does what and why? Why are there at least three Discord servers? Which Telegram groups are there? What is discussed where? Which topics were discussed, which projects presented in which meeting that I can see the recording of where? What’s the difference between “Governance Oversight” and “Circle”? What are the strengths and weaknesses of the different community portals to Catalyst? What improvement proposals on which level (assessment guidelines, radical alternatives to assessment, voting, vote evaluation, auditing, …) are there? Where is the discussion on them? Is it a discussion on the current execution or a discussion if we should maybe do it totally differently? Who decides? Why? …

In the end, this should all be answered on the central hub website above. But until then, an agreed upon introduction between all of the plethora of community initiatives, websites, councils, … would already be good.

1 Like
30

Many good points have already been mentioned here, but what has always surprised me personally is the fact that the proposal reviewers are anonymous. That, along with (my feeling) that people often don’t seem technical enough, is becoming more and more of a problem. Why would someone who writes a devastating or overwhelming review hide behind anonymity? You have to give people the possibility to see who rates what and how often.

5 Likes
31

Nice post!!
I also wondering if it is doable to implement a cost for publishing the proposals in order to avoid spamming and make sure there is a real commitment behind.

Besides I believe dReps will do more bad than good.

2 Likes
32

Imo CA (reviewers) should be elected by the community and indeed be known instead of being self-nominated and free to chose the project to assess.

2 Likes
33

After good discussions – Thank you for that! Please continue! ­– I see (not surprisingly) other proposals – in this thread and in the other spaces linked. Some might be combinable with mine, some are alternatives.

I’d like to get an impression on two (meta) questions that drove my initial post and the core demands at its end:

How urgent do you think improvements are?

  • Catalyst is fine as it is.
  • Catalyst needs ever-ongoing improvements, but we can continue the experiment and do them step by step.
  • Catalyst needs significant improvements, before the next fund is started.

0 voters

How should improvements be found?

  • Catalyst is fine as it is.
  • The current approach – engaged community is working out improvements and Catalyst team implements the consensus – is working fine.
  • Improvements can be worked out in the engaged community, but all ADA holders should vote on them in the end.
  • We need a broad discussion about improvements in many different groups and a vote by all ADA holders in the end.

0 voters

(Of course, this will be just people active in the forum who find this poll. And, of course, the result depends on my choice of questions and answers. But I hope it can at least give an impression.)

3 Likes
34

Great. I should vote but I don’t. Why? Too complex for the reasons you gave above, and frankly just takes way too much time.

Strongly believe there have to be better metrics published showing performance on previous funding rounds so there is a feedback loop and the system learns. Which projects did well, which bombed, which teams did not actually deliver against commitments and can no longer be trusted.

Building an open source VC is a very complex task. We should not expect to get this right straight out of the box!

3 Likes
35

Yeah all of this would be great. I think they started essentialcardano.io to at least fill the information part of this, but so far it hasn’t been utilised that much (IOG hasn’t marketed the website at all). Someone just need to do the job and submit but

You should’ve seen how it was in fund 6-7… there wasn’t even an Google sheets gathering all funded proposals, links to the guides, links to reporting form in one place, links to reports. Nor was there a mailchimp “old newsletters”. And I’m not even sure if the YouTube bio for the (1+ hour long) town halls had all relevant links. Now the Town halls are usually less than an hour, sometimes just 40 min even.

The Catalyst team is 20+ people strong but the community/communication team is like 2-3 people. That’s not enough. I have absolutely no clue as to why they didn’t expand it back in January.

One thing that might be worth mentioning for those who don’t know is that the Product owner of Voltaire suddenly and unexpectedly passed away in January.

Sorry if I came off as something like “join in and do something or shut up”. My thesis is that what we need first and foremost is good people actively contributing. Every chance I get, I encourage pick up a shovel because I see it as a necessity. And whether or not it is worth continuing despite the many flaws is a much more difficult question which I haven’t analysed enough to take an informed stance on. I’d have to consider the success of the ~1000 proposals that have been funded. There’s been plenty of amazing libraries and infrastructure that has been built out thanks to Catalyst. Sure, there’s waste but what’s the metric for success? 80% of Google employee projects fail. If we stop, what happens to our developer infrastructure projects? No other Web3 ecosystem has gone from 0 projects to more than 1000 in 12 months. There’s a real cost in slowing down, so trade offs need to be made.

Either way, IOG did say that they’re taking a longer break between rounds this time (finally they listened to that piece of feedback at least…) to properly have a look at the flaws etc. no start date for fund 10 has been announced.

Moreover, I don’t see how this idea of creating tasks to produce long form documents/research/alternative solutions is in any way orthogonal to “the funds belonging to all AD holders”. Perhaps I wasn’t clear but the idea is then that all of this would be disseminated as far and wide as possible within all possible Cardano community groups for feedback and input, before being put up for a vote and then hopefully implemented by IOG. Oh any and all changes ultimately require communication/collaboration with IOG. The AIM (cardanocataly.st) got access to the ideascale API after/around fund 7 I believe, and so has been able to provide a lot of value to the community since! Why hasn’t is been done before? No one has had time / been willing/been able to get paid to put in the effort to do it.

The treasury I mentioned that finance the remuneration for the people who do the research/develop the alternative ideas is based on a Catalyst proposal in fund7 (I think).

Projectcatalyst.org is run by PACE (George Lovegrove). It is an open GitHub repo which anyone can make pull requests to. It’s community held. Again, there’s just no one who has time to do it. This will also be one of the Dework tasks

I think this site would be the best candidate as it already ranks quite high in SEO terms.

I’m 100% an insider and I believe Im not an outsider in terms of views. There’s very little substantial content I disagree with in this entire forum thread, except possibly that we should pause Catalyst fully until it is “fixed” (wherever fixed means). Based on what do you get this idea that there’s misalignment?

I don’t like how you treat the Catalyst bubble as one big homogenous entity. The star rating is based on subsets of PAs, and is by no means a reflection of any type of consensus. Getting 4-5 stars doesn’t mean the project is great, it means the proposal follows the guidelines and is complete. dcSpark spent very little time on their proposals, got low stars and got funding because they have credibility. That’s totally fine in my view, the PAs did their part (in this case).

I tried to push for creating “Project Catalyst Improvement proposals”. I didn’t have much time to pursue it further. I think it’s a great idea and approach.

Thanks! These are great and would fit well as some Dework tasks (feel free to add them as tasks as you join and explore Dework, and if so, add me “Simon (Salen)” and JeremyB as reviewers if you end up doing it)

Decentralised is not a well defined concept and I don’t want to linger in this but yeah you’re right. I was a bit sloppy with the word decentralised. I think permissionless is more appropriate here. It is not under IOG nor the Cardano Foundation or Emurgo, the three big player in the Cardano space which, in my mind, means that it contributes to increasing decentralisation of power (informal).

2 Likes
36

I love this post. It is articulate and really highlights many of the problems being experienced with Catalyst. The issues you have highlighted @HeptaSean are likely a reason why many people who have become disillusioned with Catalyst (myself included), are not engaging with the process as much as they were in the early days when we were enthusiastic about the potential for change that Catalyst had.

Firstly, the point about a central source for everything Catalyst (i.e. an official website), was something I also championed in the early days. I have been part of Catalyst from the start, yet I still don’t know where to find everything. How then does someone new navigate this complex system? While I support the idea of decentralisation wherever possible, there are some capabilities that are disadvantaged by decentralisation. So having a “decentralise everything” approach, without ways to mitigate the problems it introduces in some areas, doesn’t work yet.

I couldn’t agree more about the PA/vPA process and therefore have nothing to add.

Tooling is also an important consideration. Given that IOG are still largely steering how Catalyst is executed, I believe the onus is on them (after all this time), to improve how access to data is made available.

I think ranked-choice voting is definitely the way only to go. The biggest challenge to this is the volume of proposals. I therefore suggested we test a multi-round approach to proposal funding (Community). This would allow the hard core voters (i.e. those voting now) to vote in round 1, to reduce the number of proposals that need evaluating. Prior to round 2 voting commencing, any proposals that have made it to round 2 that the community have major concerns over (e.g. Daedalus Turbo), can be discussed ahead of real money being distributed and if the concerns are legitimate the voters in round 2 should hopefully vote against it progressing to round 3. By the time we get to round 3, the quantity of proposals should be small enough that a larger number of ADA holders participate.

I would be keen to jump on a call with anyone interested in discussing this further. While text based mediums like this, discord and telegram are fine for casual discussion, they are too slow moving to make any real progress.

2 Likes
37

Great discussion. Few moments I figured to add some comments.

Heya - what sort of CSV do you have in mind?

Every newsletter has a directory of ‘past issues’ - it’s in top left corner.

Yes, they did. Here is a quick example from June 2021 - Fund5 TH

What are you missing in communication/community work?

‘Cooldown’ period between funds was introduced already in Fund8. In Fund9 - we’re building upon that experience.

It is not community held, it’s exclusively held by George afaik.

This is a standard practice in academic peer review circles. The basic premise is that it should shield reviewers from targeted attacks - and without that fear of retaliations can present their true opinion. Tho, system still is missing credentials that would allow us to level up - currently it is truly permission-less. Introduction of DIDs is fundamental. We’re very close to introducing some elementary DID on vPA level and based on learnings extending it to PA cohorts as well. Further, adding reputation model should help improve access/eligibility to write reviews based on the past performance. I recommend checking out some of the most recent work in anonymity in review work here by NASA. Interesting concepts with some great insights.

In coming month - long time coming - there will be an ‘official’ home for Project Catalyst - as in v1 landing page from which we’ll be iterating on. This will help with providing contextual framework in onboarding as well as keeping up with existing affairs.

One of the key deliverables is publishing more information from Jormungandr chain and ultimately - make it readable to people as well. ‘Catalyst Explorer’ is in the planned works and I think that will help a lot to clear up conversations.

It is generally accepted that Ideascale does not serve our present day needs as well as it should. It has served us to date but lining up gradual transition is underway.

Outcomes portal - part of the imagined Catalyst home page - will be serving this gap. Much work is being done to develop and deploy this.

Lastly, wrote this a while back. May be of an interest.

3 Likes
38

The basis for your result PDF would be a start. Plus number of wallets Yes and number of wallets No.

Not undisputed: https://en.wikipedia.org/wiki/Open_peer_review
(We already do 2. and 3., maybe 1. should also be there. Especially since reviewers in academia are invited based on the editors thinking that they are qualified to do it, which is very much not the case in our system.)

And it would be less of a problem if the reviews wouldn’t have such a huge impact on voting results, which they are not really made for:

2 Likes
39

Understood. There is a CSV output included with each PDF result directly. Perhaps, making it more visible would do a better job. I’ll take a note of it. You can find the historicals in the top left table here as well.

Screenshot 2022-10-10 at 11.59.28
Screenshot 2022-10-10 at 11.59.281518×1126 96 KB

3 Likes
40

Number of wallets Yes and number of wallets No would be good. Seems it was there before Fund6. Can’t see why it was removed.

41

The point is, there hasnt been and still isnt an answer on long term goal of the project wrt it’s sustainability, without a commitment of codebase not being thrown away, and addressing it on public chain - NOT going through some intermediate folks - can’t expect some sort of building from many.

That’s actually worse, instead of opening up into public chain - it is doubling down into chain being run by IO and everyone else being clients. Compare explorer.cardano.org and cexplorer/cardanoscan to see what this difference means in reality

That’s good to hear, hope it addresses the topics raised (those are pretty basic ones) - but also, wonder where’s place/phase for input on this transition process?

Most importantly, what can be done to put a halt for catalyst project, atleast address a big portion of process - before continuing to round 10 and celebrate stats?

3 Likes
42

We can’t tell how each specific wallet voted. Whether YES or NO - because vote is conducted in a private mode. We know how many wallets participated (vs. just being registered) in the outcome of the vote for every proposal - but not their actual decision.

43

Is that Jormungandr-based system documented anywhere? It somehow contradicts the reason for using a blockchain if transactions cannot be tracked (even if that blockchain is running permissioned behind closed doors for now). Or are you using one of the end-to-end verifiable secret ballot systems discussed in academia? Which one? None of them seemed mature enough that I would have proposed it as a solution to @Terminada’s request for a secret ballot above.

Also: Is that supposed to change for dReps? Or will we never know how “our” dRep (if we choose one) voted?

2 Likes
44

It’s not meant to be the final solution. It’s a step towards more transparency. I hope it can be appreciated and elevate conversations from ‘assumptions’ to working with ‘data’ - that’s the positive outcome in the interim imho. One of the elements may be releasing archived data from previous active instances. Mind you - Jor is not always on afaik. That may allow for more broader ‘explorer’ access to data that will be ‘dumped’ in one go to scrub over.

I am lobbying for a concept of Catalyst testnet that would enable development outside of core premise and speed up iterations/innovation. I think there’s some good movement on this front to make it happen. Let’s see, it’s early.

Putting a complete halt can have a lot of side effects as well - Catalyst is an organic constellation of thousands of people. Put it in park mode for too long - you’ll loose a lot of resource that makes the wheel run. Imagine, how long can you hold your breath without oxygen coming in? I think there needs to be some consideration to that. Bootstrapping has been the most challenging part to date and there are a lot of moving parts to date that make it happen. It is far from ‘just’ being IOG - and in that sense we need to find the right balance of sorts.

Tho - number of measures have been put in place already to slow down the pace and allow for processes/tooling to catch up. Namely, cooldown periods, longer timeframes in each stages, cap on total funding available (both in dollars but also in ada). If truly needed, more drastic measures could be introduced but again - we have to be mindful at what cost.

Having said that, I don’t think F10 will start momentarily and break may be a little longer than what we’ve seen to date. I trust that will help provide more time resource to help address varied issues. We’re actively listening and I hope next steps will meaningfully highlight that notion.

dReps final vote outcome is planned to be publicised. There will be two voting plans - direct voters and dReps. dReps will have public outcome post voting period. As usual, ongoing votes are currently private. Re implementation, let me see if documentation is already somewhere or not yet. It’s one of the things that we’re seeking to level up and host in one place - e.g. upcoming Catalyst portal.

1 Like
45

I stand corrected. My bad

Oh, I actually thought there were more people than George part of PACE. But either way, the important point is that the greater community could contribute to it even though he is the custodian etc

That was great but it was merely a pause, not long enough to properly sit down and work on changing documentation before the next round was started, hence all guides were still quite rushed. It was very much a “get the new PA guide” out last minute. Definitely no time to circulate it around the wider (i.e. not just the most engaged PAs) Cardano community

46

I hear you and 100% agreed.

47

I think the main thing being the lack of an official website/landing page where everything needed can be found. There are often so many questions like when do rewards come, how are they calculated, how do I know what my PA ID is, how good were my assessments, what did vPAs comment on my assessments, how does the onboarding process look like after winning, when are rewards distributed, how does one change parameters to catalyst, where are current problems gathered, which are the currently ongoing Catalyst “community infrastructure” projects and what’s their role (basically an overview of the Catalyst ecosystem).

Then there’s the issue of just the wider Cardano community’s understanding of Catalyst/communication towards them. I am not the best person to ask here but I think it is quite illustrative that @HeptaSean who started this used the term CA rather than PA, just as (possibly) a tip of some iceberg of misunderstandings of the process!

I think there’s very little concerted marketing and PR effort to get people to really participate. There’s a post here and there but not the repeated “spamming” and constant reminder that 14 days left to register, 7 days left, 3 days, 1 day, last chance… or “here is a video with tools you can use”/“here is a video explaining how everything works”! I am hoping that community calls should be pushed and marketed through IOGs main channels way more than it is now, through IOGs main channels.

Like why aren’t the Catalyst Town halls announced on the IOHK main twitter? This small change would, I believe, change the narrative away from Catalyst being a “sub-culture” (as I think someone called it)