🏅 Top traders

Takeaway from standup today: undoing a resolution is a thorny feature that we can't build immediately, and it would also be too tedious to do this in an ad-hoc way. If we do implement undoing resolutions (a good feature for users on its own) we will likely N/A this market.

On the positive side, the reporting works, and the user is now flagged as unreliable, so he shouldn't be able to do this again.

@FRCassarino why don't we just ban him in this case? I could easily see users missing the little triangle

@ian I don't have a strong opinion on whether we should ban him.

About the little triangle, I'll push some changes making it more prominent.

@FRCassarino I guess I'm saying the triangle should be a notification for us to likely ban the user, in which cases the triangle isn't that important, but the notification to us is.

@ian There might be cases in which the user resolving a market incorrectly is a mistake, and not a banneable offence.

Even in this case, I'm not sure what our policy is. Let's discuss in the standup!

@ian Isn't part of Manifold's philosophy that people should be allowed to resolve markets however they want? I can imagine subcommunities that would want to be able to resolve their markets wrong "for the memes." So escalating from a reputation penalty to an automatic ban seems harsh.

@FRCassarino the community guidelines are extremely clear that this behavior is allowed. https://help.manifold.markets/community-guidelines "Market creators have the final say on what the resolution is, even if it is inaccurate."

I can understand some of the reasons for it, although I personally am in favor of allowing authors to resolve markets in 99% of cases but being able to correct obviously wrong resolutions in cases like this: /jack/poll-should-it-be-possible-to-dispu

If you are considering massive reputation penalties for wrong resolution, even up to a ban, then why? Why not just fix the wrong resolution instead? Policy-wise, it makes no sense to me to penalize/ban someone for a wrong resolution rather but still say "authors are allowed to do whatever resolution they want". Technically, right now it is not possible to undo and fix a resolution, but that really is important for both accidental misresolutions as well, and I suspect that fixing resolutions is easier to implement technically than implementing a robust reputation system that works half as well as simply fixing the wrong resolutions.

@MichaelWheatley I agree - it really sucks that this user defrauded so many, but that in part lifts up the users who have a track record of resolving markets correctly.

I think this points to the need for a sort of verification system where users that have a track record of resolving markets correctly can be made clear. There is already the badge system, but I have no idea what it means!

@MichaelWheatley Yeah I suppose there's a gray area. In the case of blatant theft (beyond a reasonable doubt) of user funds I'd opt for a ban from anything but betting. I think this can be in addition to @jack's idea, if only to reduce the misresolution case load on adjudicators

Yeah, agreed that there are meme markets where wrong resolutions should be allowed.

I think if you have a way to dispute and fix obviously wrong resolutions, then they pretty much wont happen, and so you won't really need to worry about banning people.

@jack Makes sense. We had some conversations about this after the Petrov day debacle, and the team didn't really have a united view on it.

Stephen mostly didn't seem to think this was a significant enough issue to dedicate significant development time into

I tend to agree, I think that at our current scale, improperly resolved markets are rare enough that we can easily just handle them manually. In the last month, only 4 markets with more than 2 traders were added to the improperly resolved group.

The Manifold team could just manually override improper resolutions for now. For this market, we'd simply change the resolution to YES.

Once we grow 1 or 2 more orders of magnitude, a more robust "appeal" system will be needed, but right now, I don't see the benefits. Sure, there's borderline cases, but for the most part improper resolutions are pretty unambiguous.

That was my suggestion back then. But the team for the most part disagreed; they care about "market sovereignty" more than I do. So I ended up implementing the flagging system instead.

---

My new proposal:

For now ( <30 improperly resolved markets a month)
- By default, Manifold admins reserve the right to override improper resolutions. We'll have a policy of fixing clear-cut, high profile improper resolutions.
- Creators can opt-out of this while creating a market, in the advanced section. The market page will show that the creator opted out.

In the longer term (30-500 improperly resolved markets a month)
- Implement a robust "disputes" system, like the one @jack suggests.

cc @ian @JamesGrugett @Sinclair @SG

I also think the hands-off policy has the advantage of being resistant to censorship and outside pressure. If you have a policy of letting people resolve their markets wrong, you don't need to worry about demands that you "fix" the resolutions of questions that are emotionally or politically charged.

@MichaelWheatley I think it has that advantage in theory, but in practice Mr Stone now has a big yellow triangle saying they're untrustworthy. In a scenario where an author was in the right but was facing censorship/pressure, the reputational threat could be just as powerful a way to force the author to change their resolution.

@FRCassarino I agree that right now the scale of the issue is tiny (I've been making monthly markets on improperly resolved markets, and they're <5 a month, and that's counting mistakes and such).

Once we grow 1 or 2 more orders of magnitude, a more robust "appeal" system will be needed, but right now, I don't see the benefits. Sure, there's borderline cases, but for the most part improper resolutions are pretty unambiguous.

It is true that the worse offenders are very obvious, so far, but I think there are a lot of pretty ambiguous cases actually. I've been running markets on which questions were improperly resolved, and I had to make some tough calls on them: https://manifold.markets/group/metamarkets-on-improper-resolution.

@FRCassarino So will Manifold be fixing this market?

@Charlie I don't know, it's not my call to make unilaterally, I'll bring it up in the standup and provide an update after.

@MichaelWheatley There is no grey area here. There is no gotcha. This was not a situation where you are dealing with the "spirit and interpretation of the law." And the resolver even said as such. What are you defending?

@jack Okay, fair point that there's ambiguous cases.

Why do you think it'd be a net benefit today to build a dispute system versus just having a general policy of "the admins will override markets they find have been improperly resolved"? Admins can of course consult with the community when making those kinds of judgements.

@Predictor If you decide to fix clear-cut cases, then you need to decide what is the threshold for a clear-cut case. It's not an easy question to answer.

@FRCassarino A dispute system that is run by the admins where people discuss in teh comments is totally fine imo. It doesn't have to be a big fancy feature built into the platform. As long as the rules are reasonably clear.

@jack This was a clear-cut case. The user said they are "sorry not sorry" and admitted to resolving the question wrong.

@Predictor https://manifold.markets/FRCassarino/should-manifold-override-the-intent

@FRCassarino

The Manifold team could just manually override improper resolutions for now. For this market, we'd simply change the resolution to YES.

Is that a thing you can do? I thought undoing wrong market resolutions wasn't a feature yet.

@jack No, it's not built yet, but if the team wants this I'll build it.

@FRCassarino That would be great! It would also be good to solve mistaken resolutions, which is a lot more common than intentional misresolutions.

we're actually going to discuss this in tomorrow's standup since today's ran long due to planning the tasks for the Nov 7 - 21 cycle

Another proposal: Allow market creators to opt in to some sort of dispute resolution. So if I want to signal that I'm not going to do a rug-pull, I can opt in and get a little green checkmark next to the market.

Then I want to create a market about [hot-button politicized issue where I have fears about censorship], I simply don't opt in; I retain the option to make a market backed by reputation alone.