🏅 Top traders

The community guidelines have changed a bit since the previous discussion. But they don't seem to have become significantly clearer overall.

For example, there was recently a debate over whether it was in accordance with the rules to manipulate someone's profit in order to force a market to resolve YES. Profit numbers are a pretty core part of the site, and there are lots of incentives around making it go up or down. If something that important was left ambiguous, I don't think this can be called a clear policy.

As another example, it's still uncertain whether large scam markets such as those of Mr Stone and Alex Rockwell are going to be re-resolved to N/A. The community guidelines just say "we might do it", with no indication of how we can actually know whether they will do it. Manifold admins have given similarly noncommittal answers in discussion in comments and on Discord.

So I think this is still a clear NO.

I don't think my criteria are unreasonably strict here; I just want a policy that is clear to the point that we don't have large fractions of site users with mutually-contradictory interpretations of the policy, and we can all generally agree on what is and isn't allowed.

I'm confident that I could write up a policy in under 5000 words that addresses all of these situations comprehensively enough for me to be comfortable resolving this market to YES.

Please provide any final counterarguments you may have now, otherwise I'll resolve the market in a day or two.

At one point there was a market of the form "will this other market be subject to moderation" and it sat near 50% (post statement). I consider that good evidence that the policy wasn't clear.

If it's still possible to create such markets, I have no objection to the planned resolution.

@ScottLawrence otherwise, I think I disagree with the proposed resolution. But I would not consider it dishonorable, and have every intention of continuing to participate in @IsaacKing markets.

@ScottLawrence This is a great point and piece of empirical data. However, personally I still think the Manifold guidelines are basically as clear as could reasonably be expected. Some ambiguity is allowed (as Isaac said in the market description) and the question is how much.

I think Isaac's interpretation would be considered extreme by most market participants, based on the earlier comments which were all in favor of "yes it's clear" IIRC. I don't think it's dishonorable to have an extremely different view than others and to stick by it, but it certainly makes me cautious about Isaac's other markets that involve subjectivity.

@jack Yeah, I understand that way of thinking. Honestly, when I found out that the given statement wasn't "good enough", I was initially pretty annoyed. But upon reflection, I think we're just all very used to T&C-type statements being unbelievably vague, and deliberately so. I agree that the guidelines are far clearer than I'm used to expecting, but to say they "are clear" isn't a relative statement, and that should be a high bar. (Whoever wrote those guidelines deserves serious props for doing a great job, of course---I don't recall ever seeing better.)

I'm still betting YES based on P(Isaac caves | the guidelines are further updated) being greater than 25%. But as things stand now, without evidence that recent clarifications have made constructing ambiguous cases at least somewhat difficult, I hope Isaac doesn't cave. (Also because, given the controversy, I enjoy making the point that I think this is the right resolution despite me being a major YES bettor.)

As far as being cautions about Isaac's other markets: this gave me confidence in them. I expect those markets will be resolved pedantically, with an eye to what words actually mean, rather than what we're used to pretending they mean. That it's become conventional and expedient to pretend that easily-distorted T&Cs are clear, isn't really Isaac's fault!

(Reasonable people can disagree with my beliefs about T&Cs. I do feel quite strongly that it's wrong to call this resolution disingenuous---the market was obviously going to be a judgement call from the start.)

@ScottLawrence I can see your viewpoint but I think words mean what readers think they mean in context, not what the dictionary says.

I agree that the resolution does not appear to be "disingenuous" or "dishonest" and I'm opposed to those particular labels, but I would personally agree with "inaccurate" because I think a supermajority of reasonable predictors reasonably expected a different outcome.

And the problem could have been avoided if the market description included a few decent examples of what would and would not qualify. The market description is pretty long and detailed about the background and context, so it's not like it would have been a huge amount of effort in addition, I think.

Sold all my shares as the current planned stated resolution is one I consider dishonorable and inaccurate. I will be forgoing the creator's markets in the future FYI.

I'm curious: do you think it's possible for someone to post a market that's of a similar level of subjectivity to this one, resolve it in a way you disagree with, and still be labeled as "honorable" by you? In other words, is there some aspect of the general way I've handled this market that leads you to feel I behaved dishonorably, or is it just the object-level fact that I you disagree with my interpretation of the subjective criteria I originally laid out?

@IsaacKing

"This market resolves to YES if by the end of the year we have a clear statement about when Manifold will resolve someone else's market or delete or edit someone else's market or comment. "

This happened. Manifold released a lot more than just a statement, they made an entire policy. Then the release of the policy caused this market to shoot up to 93% before you posted and said that for reasons it would still resolve NO.

Note the existence of the word "clear". You also left out the rest of that paragraph, which further clarifies that I had certain standards in mind:

"Some ambiguity and subjectivity is of course unavoidable in these sorts of policies, but it should at the very least tell us what can happen to our markets/comments against our will and what sort of things we need to do to avoid that happening."

You can disagree on exactly how clear the policy is, but claiming that the criteria were unambiguously met just because Manifold released a statement, without any reference to what that statement contained, is just flat-out wrong; that is not what the market description said would cause this market to resolve YES.

@IsaacKing And I strongly disagree, imo the market description was met.

"it should at the very least tell us what can happen to our markets/comments against our will and what sort of things we need to do to avoid that happening."

The community guidelines tell you both:
- What can happen to your market and comments (delisting, deletion)
- And lay out rules and even examples for what is subject to moderation

Your statement even supposedly allows for ambiguity and subjectivity, but then later go on to not allow it.

Just laying out what can happen to a market doesn't mean I know what will happen. Similarly, the simple existence of rules and examples does not mean something is clear; it depends on what those rules actually say. Consider for example the following policy:

"We may or may not do anything at all to our user's markets at our sole discretion, including but not limited to delisting them, deleting them, reporting them to the FBI, editing them, and/or overlaying smiley faces on top of them in the UI. For example, if one of the Manifold employees really doesn't like the letter 'P', they may choose to edit all markets starting with that letter to instead start with the letter 'Q'."

That policy satisfies all of your criteria, but I think all reasonable people would agree that it is not a clear or useful policy.

My idea of a "clear policy" is a policy that allows me to know with high confidence what will happen to the vast majority of markets that are posted. The current policy doesn't do that. As I pointed out a few weeks ago and Scott just mentioned again today, this isn't some extreme hypothetical; there have been existing markets where Manifold's users had no good idea whether they'd be subject to moderation. If the users of a website based entirely around accurate prediction can look at a policy, look at a market that was not deliberately engineered to fall through the cracks of that policy, and still have no idea what moderation will happen to that market, that does not strike me as being a particularly informative policy.

Also, to be clear, I do take some responsibility for this disagreement. I should have predicted that a policy like this would be posted, since this is the sort of superficially-helpful-but-once-you-look-at-it-closely-it's-nearly-meaningless language that a lot of company PR departments tend to use. In the future I will try to hold my markets to the same standard to which I'm holding Manifold, and make sure their resolution criteria are clear enough that it's unlikely this sort of disagreement will happen again.

@IsaacKing Just to disambiguate one point: it's (probably) always going to be possible to construct an edge-case market. I don't expect a "clear" policy to be resilient to attempts to evade that policy---that's a hard problem! But it shouldn't be so easy to find one in the wild, and casually create a market that sits at 30% or so for days (if memory serves).

I've commented elsewhere (in the context of physics markets) that creating good markets is extremely hard. It's not really visible here, because most bad markets are still able to resolve without drama. That creates the impression that only the small percentage of markets for which resolution is contentious are truly bad, but that's not the case: any market which could plausible have a contested resolution is bad, and that's quite a lot.

Metaculus has a culture of spending days/weeks debating the language of a market before it goes live. This mess right here is why. (I don't think that culture is a good match for Manifold, FWIW.)

@ScottLawrence I completely agree. (Hence why I included the "not deliberately engineered to fall through the cracks of that policy" disclaimer in my previous post.) It's never going to be possible to remove all ambiguity when it comes to rules on general human behavior. (Hence why court cases are still going strong 200+ years in.) The best we can do is cover some large fraction of cases; I think 99.99% would be a reasonable target for a policy like this one. Manifold could keep track of how many markets result in controversy due to moderation, or how many include disagreements between team members. If it falls beneath that number they can then consider adding more specificity to the policy.

(If your reaction is to think that 9999/10,000 seems much too high, consider that the vast majority of markets are completely unobjectionable, and quite obviously so. I don't have an easy way to see how many total markets have been created on Manifold, but I'd guess it's in the 1000-10,000 range, and maybe ~4 of those have included controversy over moderation thus far.)

I just came across the first instance I know about of Manifold removing a comment, though it happened before they published their guidelines, so it's not directly relevant.

https://manifold.markets/LivInTheLookingGlass/will-feminizing-hrt-make-me-like-pi

I don't know what the original text of the comment was.

@IsaacKing as everything stands now, are we at a YES or a NO?

@Charlie I think we're currently at a NO, but I'm open to being convinced otherwise. For example, if someone shows that it would be infeasible to write a clearer policy without making it dramatically longer and more complex.

@IsaacKing To clarify, is this market going off of your definition of “clear policy” or the average commenter’s definition? I’ve been betting under the assumption that we’ll be sticking to the characterization of “clear policy” that you’ve elaborated on in the comments, but I’m just wondering how you would resolved this if @jack ’s poll is a near-unanimous YES.

@Charlie Good question. I plan on resolving this according to my judgement and the standards I've explained below, but if it seems like everyone except me feels that the policy is clear enough for them to be happy with it, I'd take that as evidence that I might be holding Manifold up to unreasonable standards and should perhaps relax them somewhat.

I don't think Jack's market is going to accurately reflect community consensus, as there's no real incentive for people to vote in it. I'd be more likely to be convinced by several people explaining in words why they think I should resolve this market to YES, and no one else disagreeing.

I also don't think my poll is particularly useful on its own - it's more meant to help put a number on the discussion comments, and I think these sorts of polls work best with both of those elements together. Here we already have a very well developed discussion, and clearly people are trying to count up how many people think it's clear vs unclear already.

I also think that these polls do generally work reasonably well in my experience - there are cases where they can fail miserably but the usually work fine. As some examples: https://manifold.markets/ManifoldMarkets/plastic-straws-should-be-banned or https://manifold.markets/ManifoldMarkets/universal-basic-income-should-exist.

The main thing I wanted to get a quick numeric estimate of was how much of a difference there was in the community's vs Isaac's judgement of how clear the policy is. I was expecting the community to be higher than Isaac because Isaac's criteria seem much stricter than most of the other commenters here. I was surprised to see that the two markets have been about the same.

Made a related market:

https://help.manifold.markets/community-guidelines
New guidelines are now out!

Let me know if there are any questions or concerns.

@DavidChee looks pretty dang clear to me.

Potential opportunity to borrow from ACX with the "will Manifold accept my argument for why I should be unbanned?" thing.

@DavidChee I appreciate the writeup and link!

This is a useful set of guidelines to have. I'm not convinced that it qualifies as clear enough to resolve this market. For example:

"this will almost always be a last resort after multiple warnings"

"Almost always"? So when will it happen without warnings being given?

"The below scenarios will be delisted if possible, but in some cases will have to be completely deleted."

In what cases?

"Markets and comments made with malicious intent to harass an individual or specific group of people."

What does this mean? If I make a market on whether the president of the US will break a promise in the next month, is that harassment? What if I do it for someone I know personally? What if they don't want the market to exist? The term "harassment" is frequently used to mean "someone did something I don't like". As written, I could come up with an argument that nearly any market is harassment.

"Once resolution criteria are met users are expected to resolve their market promptly."

What does "expected" mean? Are there penalties for creators who don't do this? Or is this phrase intended just to mention the existence of a social norm?

"Manifold may step in and resolve markets on their behalf."

"May step in"? We already know they may step in! I'd like to know when, specifically, they'll step in. Saying "we may step in" doesn't provide the community with any information we didn't already have.

" Manifold Markets reserves the right to unlist or delete any comments, markets, groups or accounts we see fit."

They reserve the right to violate all of the above guidelines, without providing any explanation of when or why they might do so. I understand Manifold might need this to be there for legal reasons, but it certainly doesn't instill confidence that we can trust these guidelines to be honored, nor give us any indication of when or why that might happen.

Overall, while I appreciate having this set of guidelines to refer to, and I think it's better than nothing, it's incredibly vague. Some parts, like the section on multiple accounts, are written specifically enough to give us a meaningful sense of what sort of actions aren't allowed. But much of it is full of weasel words that serve to give a false impression of being definitive statements, like "we may do X", "we don't intend to do Y", or "users are expected to do Z".

I'm open to being convinced otherwise, but at the moment I don't think this qualifies as a clear policy on what behaviors are permitted on the platform and what, specifically, will be done about the ones that aren't.

IMO it's more detailed and clearer than I expected and I think it helps clarify the moderation policy immensely. For example, regarding the "resolving markets" section, I think the individual bullet points aren't intended to be read separately; the section as a whole seems pretty clear to me: "Once resolution criteria are met users are expected to resolve their market promptly." -> community and/or admins can remind (or warn) you. "If clear resolution criteria are met and 30 days pass without hearing from the creator, Manifold may step in and resolve markets on their behalf." -> Manifold will resolve on author's behalf at this point if they want to, but does not commit to doing so in all cases (because that would be a lot of work). The 30 days part is a pretty clear answer to your question about the timeframe imo.

The overall point of all the "mays" is that moderation cannot be guaranteed. That's just a fact of life, I don't think it's reasonable to expect anything different. Some things can be automated, e.g. automated resolution of abandoned markets could (and imo should!) be built, but there's no way you can have automated detection of "Markets with titles containing spoilers." so it's fundamentally got to be a "may".

I do agree that aside from those, most of the parts you pointed out could benefit from being clarified further.

@DavidChee I plan on delisting markets that appear on the trending page with titles like: "Will you use this market for your betting streak?" but not sure exactly how to describe that type of market, have any ideas?

@jack Even if it's clearer than I expected, that doesn't automatically mean it meets the bar I'd like it to meet. The current market resolution policy doesn't tell me:

• When (if ever) reminders will be sent.

• Under what conditions Manifold will step in.

• Whether there are any consequences for repeated delinquent markets.

Note that there are a bunch of markets that closed months ago and the Manifold team has not resolved, so we already have evidence that the "we may step in" is not "we will almost always step in".

Here's an example of a market-resolution policy I would consider acceptably clear:

"If a market has closed but not yet been resolved, we will send its creator a reminder email 5 days after close, and another email 20 days after close. If 30 days from close it remains unresolved, that market is marked delinquent. Delinquent markets are subject to third-party resolution. If the Manifold team believes there to be a clear 'correct' resolution to that market, we will make a comment confirming that that the community agrees with that resolution. (For these purposes, 'correct' means 'what the creator intended the answer to be', not 'what the Manifold team believes in a vacuum to be true'.) If they do, we will resolve that market. If we cannot arrive at a consensus on what the market's resolution should be, we'll resolve it to its probability at close time. We aim for this to take place within 5 days from the market being marked as delinquent, but it may take longer if the Manifold team is busy with other things."

@IsaacKing Regretting betting in this market as this should be resolved YES very clearly IMO. I would consider a NO resolution at this point to be a dishonorable resolution.

@SneakySly There's no reason I would resolve it to NO now; that would only happen at the end of the year. The question is whether I should resolve it to YES or leave it open.

My criteria were:

"Some ambiguity and subjectivity is of course unavoidable in these sorts of policies, but it should at the very least tell us what can happen to our markets/comments against our will and what sort of things we need to do to avoid that happening."

Has that been met? It's close, but I don't think it gets there. The worst offenders are:

• "Content that [...] facilitates illegal actions."

• "Markets that are hateful in nature including those that discriminate based on race, gender, sex, or disability status."

• "Markets that could incite terrorism, violence, or abuse."

• "Markets and comments made with malicious intent to harass an individual or specific group of people."

These are phrased very vaguely. They could be interpreted so narrowly as to apply to nothing or so broadly as to apply to tens of existing markets, depending entirely on how the moderator chooses to read them.

Does my market on whether COVID was created in a laboratory "incite violence" against Chinese people? Such an interpretation would seem ridiculous to me, but thousands of Americans would likely believe it qualifies, including many who write for major media outlets. Does your market on whether a major US city will be hit by a terrorist attack "incite terrorism" or "facilitate illegal actions", since it provides an incentive to commit terrorism? It certainly shouldn't count, but if a Manifold employee were to delete it and point to this policy as their justification, it would be hard to argue with that. Are markets asking whether Donald Trump will be indicted "harassment"? Does the market that asks whether someone's child will be a boy or girl discriminate against nonbinary people? It depends entirely on the views of the moderator doing the moderating.

Policies like this are a common tactic of organizations that want to leave their options open. They carefully craft wording that people are likely to interpret as in line with their personal beliefs and not realize are actually highly ambiguous. (Sort of a variation on the Barnum effect.) Then the organization just goes ahead and does whatever they wanted to do anyway, and if they encounter any pushback, they can point to their policy and say "this is in line with the intended interpretation of our policy".

(I'm not saying Manifold employees got together and hatched a nefarious plot to mislead their users; I'm saying this is something I've seen other organizations do, it's an effective strategy, and I think even if it was unintentional in this case, the wording of the policy is going to lead to users interpreting it as being more specific than it actually is.)

@SneakySly Oh hey, a perfect example just got posted.

https://manifold.markets/J/in-2030-will-wikipedias-article-on

Does this market violate Manifold's community guidelines? I honestly have no idea.

https://manifold.markets/IsaacKing/will-the-linked-market-on-race-and

@IsaacKing

"May step in"? We already know they may step in! I'd like to know when, specifically, they'll step in. Saying "we may step in" doesn't provide the community with any information we didn't already have.

for experienced knowledgeable members? no.

for new people? yes.

It is useful to write down such things

@M That's a good point. I was interpreting the policy from the perspective of an experienced user and wasn't considering a new user.

@IsaacKing "Does my market on whether COVID was created in a laboratory "incite violence" against Chinese people?"

A good example. I would definitely feel a lot better if Manifold clearly articulated what philosophical perspective they're coming from with their moderation - clearly denunciating the "words are violence" silliness in favor of a fairly strong free speech position would be nice.

Rereading the guidelines, I think that's where they're coming from, but I may just be projecting/wishcasting there haha.

@IsaacKing @MattP We've made some slight adjustments based on your feedback. I suspect you're not going to be fully satisfied though.

When writing these I was focused more on how the majority of users would use them; something that is concise but also clearly outlines what isn't acceptable. Evidently the latter isn't as clear as I thought, but I can't think of a way to make it any clearer taking into account the nuances that occur when making decisions about these things.

Regarding resolving markets: We plan to start actively resolving more markets manually whose owners seem to no longer be using the site and are unresponsive to @'s or emails. I added a lot more details of this in the doc. Probably do a sweep through 10-30 each week of the ones with the most locked up mana or are the oldest. The reason we only resolved a couple before was because we didn't have a quick way to do so set up but should add it in this week.

Terminology regarding "inciting violence", "being hateful", or "harassment": I've changed the wording slightly to make it marginally more clear. The fact is though that this is something that will have to be done on a case-by-case basis and will look at a number of factors such as the history of the offending user and the context. But, all the markets you guys have listed as examples don't come particularly close to what we would consider delisting (and for example under these guidelines Eharding's market that was deleted also wouldn't). The only reason we are intending to take minimal moderation action is if we think they endanger the development of the site by alienating a significant number of users.

As I've previously stated our goal is to eventually not have to do any moderation (except for things which are clearly illegal). So your most recent message @MattP does hold true. The founders particularly agree with your side and have expressed to us the clear desire to keep moderation to a minimum.

I come from a background of facing the realities of what will happen if you let a site or community go completely unmoderated in its developing stages. That's why I was a bit too hasty with deleting Eharding's market. Now that we have clearer guidelines in place things should more consistently align with Manifold's philosophy (creating these guidelines was just as helpful for us too).

@DavidChee I really appreciate how responsive Manifold is to feedback. It's nice to be able to raise concerns and get a direct response to them. It makes a nice change from many other organizations I've dealt with. :)

The clarifications you added are good to have, thank you. A few questions for you:

When writing these I was focused more on how the majority of users would use them; something that is concise but also clearly outlines what isn't acceptable.

If you're concerned about people finding the policy daunting, maybe you could have a short version for most users and link to a longer version on a separate page for people who want more details?

I can't think of a way to make it any clearer taking into account the nuances that occur when making decisions about these things.

I agree that sort of thing is definitely challenging. I personally take an approach to moderation in the spaces I'm in charge of that's similar to the approach I think Manifold is trying to go for. While I wouldn't be able to completely objectively specify every aspect of my policy, I think I could be significantly more specific than the current policy Manifold has posted. If you're interested, I'd be happy to write up a more detailed version of the manifold policy (asking you for clarification anywhere I'm unsure of your intentions), and you could draw inspiration from it or modify it to suit your needs.

As I've previously stated our goal is to eventually not have to do any moderation (except for things which are clearly illegal). So your most recent message @MattP does hold true. The founders particularly agree with your side and have expressed to us the clear desire to keep moderation to a minimum.

Why not mention that guiding philosophy in the policy? A paragraph at the top about how Manifold wants to put its users in control of the content they see and doesn't want to take any top-down moderation action unless compelled to by law would help alleviate the issue of people not knowing how broadly those policies will be interpreted.

if we think they endanger the development of the site by alienating a significant number of users

I don't find this particularly reassuring. Allowing questions about homosexuality, porn, God, China, transgenderism, Russia, abortion, etc. collectively alienate millions or even billions of potential users who don't think certain things should be questioned or certain information shared. Would banning one of those topics lead to more users on the site? Probably not, but I could certainly see a situation where it might.

We don’t consider the ability to earn mana as an incentive with regard to [markets that clearly encourage and could directly lead to terrorism or abuse].

What would qualify then?

In general, if you don't want to add more specificity in the policy itself, maybe include a few examples of markets that do and don't qualify for each part of the policy?

@IsaacKing
I'm generally satisfied now with how the policy stands, although I agree that the suggestions you make would be improvements. I will add some of these, but realistically might not take the time to make all the adjustments you desire. But I am more than happy to consider any specific copy changes you suggest if you did want to take the time to do that.


Long version + Short version

Something I considered for sure and will probably do at some point, but definitely not worth doing for now. I don't expect to be moderating more than a single-digit number of markets/comments over the next few months lol.

Alienating users
In hindsight, this statement did a poor job at encapsulating my meaning and reflecting how possible moderation will look. This will be the part of it that will have to be done the most on a case-by-case basis, but still with regards to certain boundaries being violated.
As you rightly put, a lot of things will alienate a group of people but shouldn't be delisted.
Some examples to help clear things up:
1. "Christians who believe lgbt is okay are hypocritical. The bible teaches LGBT is a sin in book..." - No moderation. These sort of markets which express an opinion/belief as a statement that could be perceived as being negative towards a group of people are definitely frowned upon but won't be delisted. Eharding's market accusing universities as being Nazi would fall into this category.
2. "Christians who do believe and support LGBT should burn in hell and are Satan lovers" - Definite delisting. Deletion if user has a history of bad behaviour or if there is any additional content in the market along those lines or if they double down in response to delisting.
3. "Jews made up the holocaust and are acting as victims for no reasons" - Delisting. Very similar to the first example, this is where personal bias definitely does come into account. That's why we wouldn't delete these sorts of markets, but we will delist them. The reason I delist this one is mainly because it's gaslighting victims. However, if they made this market and actually presented it in the form of a compelling argument explaining their belief (and didn't make snide remarks towards victims) then it probably wouldn't be delisted! Obviously what that could look like is subjective as well.

Markets encouraging/leading to terrorism/abuse
What would this include then if not Mana? Well, you may have noticed I changed it to encouraging. The way I see a market breaching this is if someone is praising previous terrorist acts or markets that lead to circle jerking that can lead to people radicalising each other to see abusing others/terrorism to be acceptable.

Anyways I didn't make any further updates to the guidelines as of writing this post. But I'll incorporate some of what I've communicated here and any specific suggestions you make that seem reasonable tomorrow. Thanks for all the feedback!