Manifold questions are resolved by their authors. Currently, Manifold gives full control over determining the correct resolution to their authors, although the admins do step in for markets that have been abandoned.
I proposed a system where authors still resolve markets, but there is a mechanism for others to dispute the resolution, in which case it is decided by the community or by admins somehow. I believe such disputes would be extremely rare. I think this keeps most of the benefits of author resolution, while removing the blatant misresolutions. I'll discuss this approach in more detail below.
Resolves to the percentage of YES votes in this poll on Nov 20. I highly encourage you to also comment explaning your thinking. I am especially interested in discussion of reasons this may be a bad idea.
See also
Long post. Tl;dr I think being able to dispute misresolutions would probably work well, and disputes would be very rare.
I think it would work better than relying on author reputations. There has been a lot of talk about building better features for people to track author reputations. I think building a solid reputation system would be very hard - it would take a lot of effort and probably not solve the problems super well. I think building a solid dispute-resolution system would quite possibly be easier and would likely make the platform work better overall.
There are many reasons motivating author resolution - as @DavidChee explains here
Many questions are subjective and oftentimes you need to subjectively interpret what happened to decide how to resolve the question. As David says "questions come up where someone has to make a call and it makes sense that the person creating the market is the best person to do so."
It allows for personal markets e.g. "Which book will I like best?"
It allows far more questions to be created than if all questions had to be resolved by the admins.
On many other prediction platforms, after an author creates a question it is out of their hands, and sometimes this means that it resolves in a way that they didn't intend, which might be good for the predictors but bad for the author (it depends a lot on the scenario, of course). Manifold's approach gives authors autonomy and control.
On the other hand, some downsides are:
Misresolutions - these are very rare, but they do happen
Authors sometimes make mistakes resolving the market that more experienced resolvers could have avoided - this is very rare, I'd estimate a couple markets a month out of all the markets on Manifold. This can be addressed by supporting undo of market resolutions.
Authors sometimes deliberately resolve incorrectly to grab the traders' mana, or to troll them. This is extremely rare, I estimate <5 in Manifold's entire existence. But when it does happen, it makes everyone extremely mad, and I think the fear of it happening may create disproportionate fear for traders.
Authors sometimes interpret a question far differently than the consensus viewpoint.
It seems like newer users might be scared away from predicting on Manifold because they might think authors are unreliable and don't know which ones are reputable.
Abandoned markets and late resolutions.
Many disparate approaches by authors to question resolution mean less consistency for predictors, although it also gives freedom for experimentation with different approaches to see what works best.
A resolution system where authors resolve markets, but they can be disputed and overriden by the community or admins in case of misresolution, would resolve one of the main issues with the author-only resolution.
The other big issue, abandoned markets, could potentially also be resolved by this mechanism, or by a different mechanism such as automatic resolution after a long enough delay (something like this). (The rate of misresolutions is extremely small. If I had to prioritize, dealing with abandoned markets better would come first)
See optimistic oracles for some inspiration on similar resolution mechanisms that allow one actor to propose a resolution, and for other actors to dispute a resolution.
One big question is, just how clear does the incorrectness of the resolution have to be for it to be disputed? The most recent example that inspired me creating this market was 100% clear: https://manifold.markets/stone/will-there-be-a-report-button-withi. But there are others that are less clear - see https://manifold.markets/jack/what-markets-were-intentionally-and-cfda0f20db47 for some examples - and if there is a dispute mechanism, you can imagine that authors who want to fraudulently misresolve would try to find a less obvious way of doing it. And what about cases where the author makes a huge profit because they interpret it one way and everyone else interprets it differently? It might not be clear whether it's a scam or not, and whether it should be overriden or not.
Author autonomy is one thing you might lose if resolutions can be disputed. I don't know how important this is to authors. To me personally, I see my markets as a collaboration with the community anyway, and take community consensus as an important input in my resolutions.
Note that you could implement this system just by admins taking action after discussing with the community. It doesn't have to be a formalized system built into the platform. The main thing is to figure out the rules and make those reasonably clear. The only feature that has to be implemented in software is the ability to undo a resolution.
1,000
3.00Authors sometimes deliberately resolve incorrectly to grab the traders' mana, or to troll them. This is extremely rare, I estimate <5 in Manifold's entire existence. But when it does happen, it makes everyone extremely mad, and I think the fear of it happening may create disproportionate fear for traders.
This is definitely true for me; any time I make a bet of more than M$1,000 I'm terrified someone will rugpull on me (though this isn't necessarily supported by data.
I liked the compromise I heard in the Discord: If a market is resolved in bad faith, Manifold admins should change it to resolve as N/A. I appreciate the arguments for Manifold not setting a precedent for adjudicating what's true but it's probably more important that trolls not be able to make Manifold suck.
@dreev Note this can still be abused for profit, albeit less blatantly:
Make a bunch of markets, ideally timed to resolve around the same time.
Place risky bets in all of them.
Resolve them correctly if the bets went in your favor, resolve them incorrectly (which becomes N/A) if the bets went poorly.
Yes, I also like the idea a lot, and I consider it pretty consistent with my proposal. I like that N/A is a way to say that this market was disputed and could not be resolved fairly.
@A Authors can already N/A markets themselves, for a lot of reasons. I don't think the ability for a resolution to be overridden to N/A would make much difference at all. Just make a bunch of markets with ambiguous criteria and then interpret them to your benefit.
I remember Augur, the decentralized prediction market system, creating a resolution system:
from https://www.allcryptowhitepapers.com/wp-content/uploads/2018/05/Augur-white-paper.pdf
Any good resolution system would need to be trusted. Gaining such trust is difficult, particularly since the system operates in a (potentially) adversarial environment.
Consider real-life examples: legal courts with all the drama, content moderation on sites like Twitter with all its critiques, voting schemes with all the potential for Sybil attacks... https://kleros.io/ is the best somewhat successful example that I know, but it's rather complex.
I'd like a system that is actually good and trustworthy, not just better than the status quo. Maybe that's too demanding? ¯\_(ツ)_/¯
@o Yes, I hid the results of the poll until the end, so that people predicting on this aren't simply predicting on the results so far. (I also stopped trading on this market myself after setting the results to private)
I see author resolution as the core feature of Manifold. It's been noted before that author resolution means the REAL thing we're betting on is, "which button will the author click when resolving this market?"
To add a dispute system doesn't fundamentally change that dynamic, it just passes the buck and makes it way more complicated ("what will a random crowd of interested parties, potentially with conflicts of interest, vote to resolve..?").
(All of this to ignore potential Manifold developer interventions, of course.)
I kind of expect any market with sufficient resolution drama to need to use the dispute system, to continue to have resolution drama as it moves through the dispute system, except then the drama would be worse/more complicated.
I kind of expect any market with sufficient resolution drama to need to use the dispute system, to continue to have resolution drama as it moves through the dispute system, except then the drama would be worse/more complicated.
That's not the case for the misresolved which inspired this discussion. Literally no one agreed with the market creator's resolution (at least not publicly). If we had a rule like "if 90% of traders agree that the resolution is wrong, they can override it" there would not be much drama about it.
Yes, what Yev said. You could use such a system to fix markets where there was a consensus viewpoint that it was incorrectly resolved, while leaving authors the power to decide subjective interpretations.
Of course, the line between these is somewhat fuzzy. But another point is: any reputation features that Manifold builds would have to figure out the same blurry lines. If we're already going to have a system where people collectively decide whether a resolution was incorrect (like the flag as incorrect feature, or other future reputation features) then we already have to deal with these hard questions, and there will already be drama about whether a market or author is flagged or not. So I'd rather put all the effort around that towards actually fixing the resolutions, rather than just flagging it as "oh yeah, everyone thinks this was wrong, so maybe avoid this author in the future".
Personally, if somehow I got into a situation where people disagreed with me about my resolution (and it wasn't a mistake), I'd much rather change the resolution (probably to N/A given the disagreement), I think being flagged as a disreputable author is much worse in my mind.
@Yev Knew that was a rhetorical mistake. Yes, it's blatantly wrong about the Current Thing, but if Manifold succeeds at growing as a platform I still think I'm right. This misresolution was easy mode, in that it's clear to everyone involved (even the author!) that it WAS a misresolution. What about resolutions for questions that are more like scissor statements? I imagine "normie" users being disgusted by obvious (to them) misresolutions, and the ensuing dispute process with the Other Team who are all confused because "what do you mean misresolved?" being far worse than just sticking to a simple hard and fast rule, even if it makes some people very unhappy.
(Epistemic status: arguing my point rather than seeking truth, but I think I'm at least not entirely wrong.)
@journcy I do think you have a good point about controversial questions, and that is the part that I think is trickiest to figure out in any system like what I proposed.
What if we take the decentralized and/or community-based approach that some of the comments below also talked about? Each market can potentially choose a different group of users to decide the resolution, if they disagree with the author. Or the author can choose to have the final say.
@jack I think it would be great to have an institution or mechanism like that around for authors to use if they wanted to, to build trust/take some pressure off/avoid CoIs, but it sounds like regulation that should definitely be styled opt-in. Don't think we need the cooling effect.
In the post above I framed it as "disputing and overriding". Does it make a difference if we look at an alternate framing: Author proposes a resolution, and there's some period for others to voice disagreement. The normal case is that the proposed resolution goes through. If there's consensus that the author proposed resolution was clearly incorrect, then it resolves based on that consensus instead. It's the same thing really, but maybe it sounds different.
@IsaacKing Yeah, I also proposed this idea in a comment below. But I think it should be opt-out, not opt-in.
Maybe I'm unusual, but if the community thinks my resolution is wrong, I am almost certainly going to resolve N/A instead. The point of the market is for people to work together to predict something, and if I as the author disagree with the predictors on what it is they were predicting, then that's a market failure that I want to fix.
@IsaacKing I'd like to see it opt-able (either opt-in or opt-out) on the level of individual markets. I can certainly imagine myself creating some markets for which I'd like to have this feature and others for which I wouldn't.
Reasons I would like the feature: To signal reliability for high-volume, clearly-defined markets. As "life insurance" for markets with long time horizons.
Reasons I wouldn't like the feature: To avoid the risk of censorship on sensitive topics. In situations where I think the community's subjective judgement may differ from my own.
Here's an example where I think the current policy is working well.
https://manifold.markets/IsaacKing/will-we-have-a-clear-policy-by-the
Isaac signalled that he doesn't plan to resolve the way some (most?) people think he should; the reason is a disagreement over what is meant by a "clear policy" for objectionable content. The resulting argument led to some good discussion of what can or can't be expected from a content policy. Creating this market gave Isaac the chance to share and debate his (heterodox) views in a low-stakes environment. (Worst case: a bunch of users lose some M$, totalling in the low hundreds, and they get some bad feelings for a while.)
Seems to me this is precisely the sort of argument we want people to have.