https://www.vulture.com/article/rotten-tomatoes-movie-rating.html

Fascinating article

The Decomposition of Rotten Tomatoes

Rotten Tomatoes’s Tomatometer, the most overrated metric in entertainment, is erratic, reductive, and easily hacked — and yet has Hollywood in its grip. “Rotten Tomatoes is something the studios can game. So they do,” says filmmaker Paul Schrader.

-

I've taken the time to look through this market in some detail. Partly so we can learn what went wrong that caused frustration so we can look to mitigate it going forward, and partly for @SirCryptomind's sake who has contributed a lot to our site.

Facts:
- The creator did trade on this market, but not a significant amount (made 392 mana) and didn't hold any positions in the 2 days up to closing. Thus, it wasn't resolved with fraudulent intent.
- The resolution criteria in the original description had some ambiguity. But the final resolution criteria resolved to does fall within it.
- There was nothing which ever directly contradicted itself (Title, description, comments).
- This was @Jackzilla's first-ever market (I don't think this should influence any decision, but is worth noting).
- The description was never updated, aside from on the final day at which point it was "too late". - The creator did engage in good faith in the comments with clarifications. Some of these comments were 11 days ago, one of which says, "I left 30 days as an option in case we stay at this weird zone where every 2 reviews flips the score" which is what ended up happening.

With this in mind, I don't think I will be N/A'ing this market. Our guidelines have always been that the creator has the final say in the market, unless we think there was fraudulent intent involved. The only exception to this is if it was unambiguously misresolved and the creator has no reasons why or becomes unresponsive.

As always, the creator can choose to N/A it if they think they did a bad job at communicating and want to avoid people losing mana for the creator's mistake. But Jack decided to resolve to NO which I think is reasonable.

My reflections from this in no particular order are:
1. How can we better support inexperienced market creators to make better market criteria.
2. How can we better communicate to market creators that they should update their descriptions and not just leave comments which aren't always seen.
3. Add rules about whether comments override title/description (although I don't think this actually solves the problem or really does anything).
4. Is it worth implementing something to protect users from rage betting their entire balance?
5. Is there anything we can do to prevent conflict between the creator and users which just ends up frustrating both parties?

I'll talk more with cryptomind to see what can be done about their situation. But I don't think that should have an effect on the resolution of this market.

@DavidChee I think having the ability to highlight creator comments would help. I respect that Sir Cryptomind is a power user with a good reputation but it’s hard to underscore how much of a horrible experience having a market was because of his and others’ manner of communication. Perhaps if a new user on their first market hits a threshold of popularity on that market, it triggers a review by admins.

If the market had 20 participants and 40 trades, light criticisms of the wording would’ve been easier to flesh out, but because the people criticizing me were clearly playing “harder” it was difficult to separate out good faith feedback from rage or bitterness. That in turn made it harder for me to provide clarity to the market. In the end I genuinely feel like people sortof hyped themselves up into a frenzy and just ignored my initial description, and expected me to have it all figured out immediately when they asked.

It’s like I queued up to a challenger league of legends game and got flamed by my team because I wasn’t playing the game right, while more sober players thought I was playing the game alright given the circumstances.

@DavidChee Inclined to agree on most points. Unsolicited feedback on your reflections:

Like often, the questions here are more compelling than the answers. Re #1 and #2, I’ve been hoping for Manifold to release interactive, accessible, and thorough tutorials. It could be integrated in the onboarding process, obviously, plus I think it’d be wise to incentivize current users to increase/refresh their skills by allowing them to create for free the question the question in the tutorial.

@Jackzilla Yeah I'm sorry you had such a rocky experience as a new market creator. I think even if you did a very bad job at resolving it, we want to protect (especially new) creators from heated comments. Need to think if there are any practical ways to achieve this.

@DavidChee wrote this in the discord suggestions earlier. The author should have the ability to pin or highlight comments. There should be a new option to sort by: Author recommended. Although, the author should update the description as soon as they can, if changes are needed, but not everyone has the luxury of time to do that. Sometimes the discussion is worth reading. A lot comments get buried under other comments.. it would help with that.

@DavidChee I'm ngl I was also somewhat aware a few days ago that this market was starting to get a bit controversial and probably should have looked at it sooner to make sure the description and all communication was all clear.

But 1. I didn't realise at the time you were a new creator
and 2. I'm watching Barbie tomorrow and was scared of seeing spoilers LOL

@DavidChee Analyzing the problem of emotional temperature: Heated comments seem to stem from fear/frustration around losing mana, righteous indignation because of “the principle of it all,” and anger at alleged bad faith. Potential solutions to these? I think more confidence in existing formal judicial mechanisms (eg star review, trustworthyish re-resolves) as well as new ones would address the righteous indignation side. Don’t think the assumptions of bad faith can be changed. Any ideas about the other components?

@DavidChee Overall, intra-community trust seems to be increasing over time and has made these issues less common.

@higherLEVELING I like your ideas about pinning comments.

@TamarSpoerri @DavidChee as a market approaches 'we're all gambling' i think people are going to lose their minds regardless of perceived market clarity - we will see lawyering of terms in general as people's egos go on the line. I was not aware of the depth of the 'gamers grinding for elo' culture that existed here, and it took me a bit looking at people's profiles to realize that this website is not just for intellectual exploration, but is also for competing fiercely. I think the competitive element of the culture here poses risks in markets as they approach 'gambling' territory.

gamers, god bless us, are assholes and find a way to be assholes when the stakes are high

I think during election season, for instance, there will be tons of new users with markets that, while clear in intention and successfully/accurately resolved, still have the potential to anger some number of power users who feel cheated out of their goals. I do not think this problem is easy to address while still making the website fun and easy-to-access for new users - part of the appeal to me was that I could just jump in and do it!

I think focusing too much on new user experience - and by extention my apparent very-bad-resolving ;) vs, the clash of 'casual culture vs hardcore culture' could be a rabbit hole.

@Jackzilla fyi I think u did much better than I would have expected from a new creator, or even semi-experienced one. You just made it really hard for yourself with your initial resolution criteria.

@DavidChee 🤝

@DavidChee Marxist theory of all Manifold controversies: The community is governed by users with the most mana. They, the ruling class, use their capital to create most of the site’s questions, and they profit off other users’ activity by providing the initial liquidity. Since those that create questions can give credence to their preferred approach to ambiguity and conflict based on their resolution, the ruling class ends up having the most influence over community norms.

@TamarSpoerri make it georgist for me what is land on manifold (maybe 'space on the front page?')

@DavidChee I just want to chime in as someone who traded a lot in this market before the controversy started. I sold my stake a few days ago expecting it to close yes. Then I came back and was surprised it was still open. (I also left a message in the discord that this market looks to be in need of moderation but I guess the whole superconductor thing was taking your time and it was to late anyway.)

This market is, like so many others, an example of failed communication. Jack had a clear idea of what they wanted the question to be about but they wrote something some (me included) understood as something different. I don't understand why manifold has this culture of 3 sentence market descriptions. Not everybody has to write a whole book about criteria like @Mira does sometimes but why not push a culture where we try to make each other's markets clearer before everything goes to shit? Almost every drama here happens for the same reason: communication failure. Take the Google market, but it's also power users like Isaac failed to define closing criteria for the WvM market. Gigacasting never put any descriptions but most markets are unclear to some degree. That Jack even put some criteria when the question closes was better then a lot markets here.

If you keep pushing the gamifaction of this app you need to address the market quality problem. The current approach is clearly not working.

@TamarSpoerri And the proletariat, who can’t afford significant losses, have to make many small trades on many markets. But their ability to profit is partially determined by how the bourgeoisie resolves markets. Rule lawyering is a tool of proletariat to challenge the status quo and gain economic, and consequently political influence. @DavidChee trying to address the class conflict of the comments by tone policing is yet another example of capitalists trying to divide the proletariat through respectability politics, and yes, I admit this is an attempt at procrastinating doing the laundry.

@TamarSpoerri as my profile says: let us seize the means of prediction!

@AlexbGoode What will the film "Bottoms" achieve on the Rotten Tomatoes Critics Score 30 days after wide release? (READ DESCRIPTION) | Manifold

look at the masterpiece of a description i put together here, i will defeat the landlords with specificity

@AlexbGoode 100% right. I think question descriptions should have fields prompting the user to explain the conditions that determine the resolution (eg, yes/no, xx%, N/A) and the close date (eg, a given time, dependent on other factors, both a time and factors).

@Jackzilla a work of art!

@TamarSpoerri yeah I've built internal prediction markets for my workplace and I sometimes force them to go through quite a bit of rigor

my lack of specificity here was not really 'not knowing how' too make claims that can survive high pressure, but more feeling like I couldn't change the description once I'd committed it, that the initial description was when interpreted in good faith sufficient, and that when I made it I had a different set of assumptions about how rotten tomatoes work/the preferences of users than ended up being true.

I think mad-libs style 'fill in the blank' markets could honestly work for 90% of the possible questions here

@Jackzilla Interesting. Experienced users, how have people typically reacted when the question they created was based on false assumptions?

@DavidChee how about a "add that to the description FFS" report button on comments that displays a flashing banner to the market creator until they do so (or a more reasonable implementation of the same general idea)?

@TamarSpoerri Well the assumption was mostly “how likely is it that the market needs MORE than 350”, I thought that 350 was the upper limit which we’d need to kick in in case reviews were trickling at 30. My initial thought was 450 to be extra safe!

@AlexbGoode Agreed!

@DavidChee

1. How can we better support inexperienced market creators to make better market criteria.

2. How can we better communicate to market creators that they should update their descriptions and not just leave comments which aren't always seen.

• Put "Update this when necessary, don't use comments for clarifications!" as background(?) text into the field "Resolution criteria"

• Promote a culture of commenting "This needs to go into the resolution criteria"

4. Is it worth implementing something to protect users from rage betting their entire balance?

No.

5. Is there anything we can do to prevent conflict between the creator and users which just ends up frustrating both parties?

Ideas (not well fleshed-out yet):

• Add the badge wordlawyer.ish. Wordlawyer.ish users can suspend a market due to unclear resolution modalities, needs to suggest adding/clarifying criteria

• Give us an option to rate the resolution modalities while the market is still open. Depending on the rating, the market could be suspended, hidden from home per default, trader bonuses could be rolled back, ...

@Primer with you mostly until that last point. please don't give word lawyers more power

@Stralor (bc imo people who hinge too much on the specific wordings of things and don't try to come at these problems from a place of sympathy and good faith are the bigger cultural issue)

@Stralor I think it’s bad faith to assume they’re not acting in good faith. ;) Seriously though, I think it’s more accurate to attribute literal interpretations to a person’s temperament/preference than to motivated reasoning.

@TamarSpoerri right, and the intent might come from a good place! but this is still the internet and we all know what that means 😏 successful communities around here have to lean into kindness or divide into cynical bickering. I believe literal-minded people have it in them (maybe moreso than most) to learn a community's etiquette, and this is a good moment to consider what shape we want that etiquette to take

@DavidChee as another new user - something I've noticed as a systemic issue is that there's alpha in looking for ambiguous or poorly worded questions and betting on them, then trying to lawyer the resolution in the comments. But this behavior doesn't really help in terms of good-faith truth-seeking. I don't have a solution, but it's something that sticks out to me after just a few days here. Maybe it's only an issue with a big influx of new users though, obviously I haven't been around long enough to see a bigger picture.

@TomGoldthwait That's definitely a thing.