Critiques and Recommendations for the Catalyst Process

Steve Aldrich requested that I collect my criticisms and recommendations for the Catalyst process and post them in “long form” so as to not get lost within the telegram stream. The following is something of a hodgepodge of these critiques, recommendations to Catalyst proposers, as well as some personal notes that all had some overlap. As such, it is a bit unstructured and isn’t all explicitly directed at a single individual or group, so take from it what you will.

Start at the Beginning

Take a step back and decide what the main priorities are. Concisely define those priorities, put them up front, and then structure everything else around that.

Maintain a Minimalist Structure

Many, if not, most people’s biggest mental barrier is getting past a blank canvas. They need some structure and recommendations to get the juices flowing and create a dialogue within their head about how the project will come together. On the flip side, too much structure can stifle creativity and won’t allow individual proposals to stand out amongst the crowd. As with most things, the best solution is somewhere between within a Goldilocks zone.

Try to Avoid Unique Phrases or References

I’ve inserted a number of examples of what not to do in the previous bullet. While they may be common place in one language, many phrases don’t translate literally into foreign languages. There may be a figurative analogue within some languages, but this relies on the translator’s awareness of such analogues and they normally will not extend to all languages.

Communication is Key

Make things clear from the onset. Speaking in vague terms allows for a diverse array of creative opinions. Sometimes, this is intended, but other times it simply produces unnecessary work for everyone. It leads to more confusion, questions about intent, and last minute resolutions to problems that become a precedent for more confusion and more problems.

When choosing wording, try imagining how a non-native speaker would interpret the text. It does not have to be eloquently phrased, but normally finding precise wording is preferred over colloquial familiarity.

Communication is simultaneously decentralization’s greatest enemy and most needed asset. Information needs to be pervasive, yet localized as much as possible. Information can be disseminated through all of the various social media platforms, but it should all stem from, and be linked to, the same few locations (ideally, Ideascale in this scenario). It’s very inefficient to have various forms of working docs that are only referenced and linked within specific channels (for example Catalyst Telegram). People need to be aware of where things are if they are going to contribute. Standards can serve a vital function here and should be debated on. Should google docs be the de-facto tool for drafts? Where will all drafts be located? What should the default permissions be so that those who need to contribute can? In the chaos of decentralization, consistency is your friend.

Intent Over Wording

When all else fails, you want users to be able to use common sense and fall back to the intent of the instructions rather than the specific wording. Examples are an excellent way of illustrating intent.

Personally, one of my most common struggles when structuring documents is how to balance concisely presenting the most pertinent information while also elaborating and providing sufficient detail for the intent and implicit assumptions to be understood.

Use intent to build a rapport with the community. If something has been rushed, there has been a massive influx of users, you are understaffed, or you tried something and it didn’t work then don’t try to hide it. Admitting that problems exist can assure users that they are not being ignored and that you are doing the best that you can.

On the other side, also be sure to provide constructive feedback to others. A small note or action has a much greater affect early on than further down the line.

Follow Through

This is sometimes at odds with intent, but goes along with communication and consistency. Once a decision is made and articulated to the group, you need to be able to commit to that. Treat it like a verbal contract. If we promised $500k to a Community Choice fund with inadequate guidelines for drafting and reviewing, then we have to be prepared to settle for whatever the outcome is rather than attempting to change the rules or make exceptions at the last moment. A hands-on approach is almost guaranteed to receive the greater blame in a no-win scenario.

Make Things Intuitive

In the present case, priority should be on function over form…but form still matters. If a funding round has ended and there is no longer a need for user participation, then why are they the first tabs visible within each campaign instead of within an archive group? If you are describing a process, then add links to more detailed explanations right there so people don’t have to traverse the entire site for the relevant information. Correct sorting algorithms to do what they say. Optimize the layout to minimize the number of actions (clicks) it takes to perform a task such as proposing or reviewing.

The rules should be reinforced and not contradicted by the form of presentation. If the rules state that all reviews without an assessment will be thrown out, then don’t make the assessment optional (much less an “add-on” option). If the assessments should be in depth, then don’t make the default box size only 140 characters. If we want proposers to provide background references for their experience, why not allow them to upload resumes and links within their profile?

Get the Incentives Right

Recognize the effects of how rewards are allocated on the probabilistic vs deterministic spectrum. On one end, if you gave all $25k to a single reviewer, this would likely attract more speculators and make them outnumber the more altruistic reviewers. On the other end, a guaranteed, or nearly guaranteed, reward (perhaps based on the number of reviewers per proposal) is typically perceived as a more fair than a lottery system, both in terms of equality among reviewers and in terms of preventing conspiracies of favoritism. Furthermore, it reduces distraction as reviewers will likely spend more time thinking about the task at hand than on the rewards and how to game the system. Reinforce the altruistic behavior you desire without exploiting the group.

If you don’t want users to game an aspect of the system, then don’t design it to be gamed. Only allow proposals to receive 1 Kudos per person instead of letting people stockpile Kudos and then dump them all on their favorite project. This would also incentivize users to use their Kudos elsewhere more often, such as within the comments section. Don’t have a project take/receive multiple Kudos just because it has multiple authors. Maybe even get rid of the Kudos limit altogether.

There Is No End, There Are Only Checkpoints

Everything is a continuous process. There will always be improvements that can be made, but at some point you have to call it quits and turn in what you’ve got. All you can do is give it your best effort. Most likely, in your eyes, it is never going to be perfect, and that’s okay. It’s not the end of the world, it’s just the end of the day.


More specific suggestions for the Catalyst team:

Ideascale Interface

  • Under “Here’s how it works” on the front page, hyperlink pages providing further in depth information
  • Create a “Roles” section where you define the Proposer, Community Advisor, etc. and clearly explain their responsibilities (provide guidelines here)
  • Create an expandable dropdown under the review categories that provides the suggested ranking criteria. For example: Category - Impact (1-5 stars) → (expandable dropdown) 1. Does the proposal highlight a problem that needs to be solved? 2. Does the proposal benefit a large group or a small subset of the Cardano community? 3. Is the idea able to scale? This would subsequently be followed by the assessment box for reviewer feedback.
  • Make the default assessment box size greater than 140 characters.
  • Allow CAs the ability to ‘undo’ reviews. If you give any number of stars to one of the questions (accidentally or not) there is no way to go back and say “I don’t actually want to review this proposal.”
    *Correctly update the tabs on the homepage to the current fund(s)…they still are set to Fund 1 and Fund 2.
  • Update the “Results” tab. This currently still only has a link for Fund 1 results (that you have to copy-paste…it isn’t even a hyperlink)
  • Correct the “Popular” sorting option for proposals
  • Add more extensive profile functionality. Allow users to upload resumes, include an “experience” section, and a “collaborators” section. Give it more ‘social media’ type features.

Review Process

  • Tailor the review questions to the challenge
  • Include an assessment question that explicitly requests how the reviewer thinks the proposer(s) could improve their proposal. If not, at least frame the statements/questions to request recommendations from reviewers. There is a (maybe not so) subtle difference between saying why “I agree/disagree” on a 1-5 scale vs how the idea could be improved to make me agree more.
  • Don’t make assessments seem optional in reviews. If you’re going to throw out ‘blank’ reviews then don’t even allow them in the first place. Make the assessment box always visible and put an asterisk next to it indicating that it is required.
  • Have the review questions ready and posted for proposers/reviewers to see at the beginning of the funding round.
  • Allow reviewers to input reviews before the Assess phase. This will provide reviewers with more time to develop their opinions as the proposals progress. Many reviews would only then have to be “submitted” during the Assess phase, while most of the leg work was done before hand.
  • Allow CAs to flag low quality proposals (the earlier on the better) so that people don’t waste effort “fully” reviewing them. This could be done as a minimum threshold of CA flags or could rely on creating/utilizing another body to review the flagged proposals (“veteran” CAs).
  • Create an “incubator” (as suggested by Greg Pendlebury) for early stage proposals…[will elaborate upon]


  • Only allow a single Kudos per proposal per user.
  • Allow individuals to remove their Kudos
  • Don’t require multiple Kudos for a multi-proposer project, people are “liking” the project, not necessarily each person involved.
  • Experiment with allowing unlimited Kudos
  • Distribute rewards evenly between all qualified reviews on a per proposal basis. I.e. for n proposals and m qualified reviews of that proposal, the reward to each reviewer for a given proposal would be R / (n * m), where R is the total amount of rewards allocated for reviewers.
  • Get rid of the “one strike and you’re out” policy for CA rewards. If a CA leaves a blank review or poorly written review, just eliminate the review from the pool for that proposal, not the reviewer for the entire fund. There could be a “3 strikes and you’re out” rule for reviewers that consistently leave poor reviews.
  • Provide an indicator for the rewards that have been paid out for participating in Catalyst. Combining voting rewards with delegation rewards confused many voters into believing that they hadn’t received their voting reward.
1 Like