Resolves YES if Manifold implements a community-based author reputation feature (e.g. a karma system or something similar) by the end of December, otherwise NO.
Examples that would count as YES include:
If the community can upvote/downvote a user's reputation, whether with pure votes or with M$, and with the reputation displayed on their profile and/or markets.
If users can vote on agreement/disagreement with a market's resolution, and information about an author's past disputed resolutions is shown on the author's profile and/or markets.
Examples that would not count as YES here (although also potentially useful features):
Allowing an individual user to add personal, private labels/notes/flags about other users
Features based solely on admin action
Features for delegating resolution
This is a follow-up to a previous market that ended in September:
1,000
3.00I'm not sure any new reputation metric will be vastly better than followers or mutual followers. Further, requiring a stellar reputation to get engagement worsens the experience for newer users, and results in attention asymmetry.
I still think one should be able to report bad actors as a way to accomplish some of what you talk about.
@FRCassarino @JamesGrugett Cool to see this work! I think flagging incorrect resolutions is a good starting point to build off of, I know the threshold of 80% correct resolutions is just a starting point, but I had some thoughts to share on how to potentially improve it.
There's a lot of different types of bad resolutions and bad authors. Going from what I think is most common to least common, based on https://manifold.markets/jack/what-markets-will-i-consider-to-hav and https://manifold.markets/group/improperly-resolved
- Authors who are slow to resolve their markets. By this metric, the main Manifold account might fail to meet the 80% threshold - which I think is an accurate reflection of author reliability but users would probably be confused if the main Manifold account is flagged as a bad author. On the other hand, DrP who intentionally trolled by resolving slowly would be above the 80% threshold and so would not be flagged, which seems like a clear miss.
- Making honest mistakes. I don't think I've seen authors who are particularly prone to mistakes, it happens to everyone (including myself). With the 80% threshold, this wouldn't affect authors who make an occasional honest mistake, so it's fine for now. But if you wanted to make the threshold work better then probably it would help if the flag system let you select one of a list of reasons.
- Resolving N/A for no good reason, e.g. new users thinking they made a mistake that could have been fixed by editing (https://manifold.markets/ussgordoncaptain/will-xi-jinping-be-president-of-chi for example) or author deciding they wanted their money back.
- Deliberately resolving wrong, e.g. troll or rugpull. Very rare, but typically people lose a lot of money when it happens. Authors should probably be flagged for even a single clear instance of this.
Doing a percentage of correct vs incorrect resolutions seems like it mostly will detect authors who are unreliable at timely resolution, but not the other types.
An interesting question would be to see what types of bad resolutions traders are most worried about and what is most likely to scare away users.
@jack There's at least one other type of improper resolution: resolving the market too early to something that's almost certainly a correct answer. For example if I were to resolve https://manifold.markets/Yev/will-artemis-i-launch-by-the-end-of-75233245efeb now, it would be incorrect, but I don't think it would be very dishonorable.
@Yev Yep that's another super common one.
Btw, general comment - I think the term "dishonorable" has outlived its usefulness - the common interpretation of it is different than how it's often used here, and so we should use a less confusing term instead. I like "improper" better although it's still not perfect.
@jack I don't think it was ever meant to be useful, it was just meant to be fun. AFAIK, the earliest use of "honorable" on MM is this pun: https://manifold.markets/Gurkenglas/honourary-resolves-honorably
@jack Hey! Agreed with all your points. I just shipped that PR, which as you say, is just a starting point. The main point of it is to let user report improperly resolved markets so we can start getting some of that data. After 1-2 weeks I'll take a look at the reported contracts and based on how that's gone, revamp the logic for showing the "unreliable creator" flag.
I also agree that we'll need to build this out further - people taking too long to resolve their markets is a bigger issue than improperly resolved markets right now.
@jack Btw this is now shipped, which satisfies this resolution criteria of this market.
"If users can vote on agreement/disagreement with a market's resolution, and information about an author's past disputed resolutions is shown on the author's profile and/or markets."
@Yev Hmm, I'm not sure but I'd think no, I think it certainly is a reputation system, but it doesn't seem "community-based".
If it the metric was something about how popular the author's markets were, that would count as community-based reputation.
Doing something, anything: easy
Doing something well: damn hard
And reputation metrics are not the kind of thing you want to do halfheartedly. Some people will certainly try to game the system; others will get upset when their reputation is lower than they think they deserve.
I don't believe any system Manifold implements by EOY would net reduce drama.
I'm betting that this is a fairly complex and large feature to build, and they're still too busy with other priorities that are more low-hanging-fruit.
However, I think improving the new user experience is pretty top-of-mind for Manifold at the moment, and avoid bad new user experience from betting on a market where the author misresolves or simply disappears is something they're definitely thinking about. See the big discussions on the Petrov day markets, for example.