thanks, I’m happy to see the emerging discussion here. I think any of the suggested changes to the review process… educating reviewers, trimming the set of reviewers to those with demonstrated committent or qualification, or eliminating the review process entirely… would be an improvement over what was brought forward in Fund 10.
TL;DR most importantly we need to restore the ability for reviews to be challenged by proposers instead of just the anonymous and unaccountable Catalyst auditors.
The potential damage done to our Fund 10 project’s same presentation in Fund 9 (an annual project designed to renew funding every year) by verifiably uninformed or spurious reviews was limited because we (and others) had a chance to flag the offending reviews. Fund 10 leaves this audit process entirely centralised to 2nd level reviewers recruited from the same population… this assumes skill, responsibility and attention levels from those auditors that we are certainly not getting.
We had spurious, uninformed, and potentially malicious reviews that were upheld in the audit process and this time there was nothing we could do about it… and it almost ruined the project, which would have left Cardano’s CIP Editing team without the bulk of its routine support.
A good part of our project statement was about the flawless execution of the same project in Fund 9 and reviewers still made the following unchallenged statements which can be seen here:
-
Reviewer makes repeated references to “the team” and “no information about other team members” for a solitary project (my own funding as a CIP editor, on a team funded from different sources including IOG and the Cardano Foundation). This easily spotted mistake on the reviewer’s part suggested “potential risks to completion” and that it was only “partially feasible” … i.e. no observation of GitHub and Catalyst prior proofs of completion with full visibility to the Catalyst and general community.
-
The longest review makes a complete inventory of all past accomplishments that confirm feasibility, concludes that our project is both realistic and feasible, and then gives it a 3/5 score.
-
Another review marks it down to 4/5 because the proposal, which he says contains detailed demonstrations of feasibility and demonstrations of trust in the Cardano community, was “too much reading”.
The greatest risk to our project in a close vote (due to another Catalyst problem identified in this series) was this low Feasibility score even after satisfying the requirements 100% for documentation and proof of Feasibility… with anonymous reviewers shooting it down without any accountability either from themselves or from equally anonymous auditors. This is not “decentralisation”… this is a mob covered by an impenetrable bureaucracy.
If Catalyst is going to keep passing out tickets for casual workers to come make casual income, either they need to be highly vetted / trained / audited or we need to stop giving out the tickets.
The same solution recommended here for Voters (education) might also work for Reviewers… but given that not all these opportunities for education will be followed, we have to insist upon visibility and community audit. Otherwise we will keep seeing small projects vital to our system challenged & derailed by reviewers who don’t even understand what the proposals are about. 
p.s. @dannyribar I apologise if I missed this in one of the “town halls” or governance deliberations… but can you please post here about why Catalyst dropped the Review public audit process in the first place?