Someone leaked the AMC 12A (a prestigious math competition) on AoPs (a math forum), discord, and many other websites. It's estimated that at least a few hundred people saw the problems. Wht will the MAA do about the leak?
More context:
https://artofproblemsolving.com/community/c5h3191445_civil_discussion_of_the_leak
1,000
3.00@JimHays Yeah, in comparison to other years MAA practically did nothing different. MAA test proctoring has been the same level of stringency this year compared to last year, especially in China. They have also shut down testing centers in the same way, there is nothing unique about the testing center shut down this year. It's just that the cheating became a lot more rampant. Them randomizing the answer choice letters is insignificant.
"Nothing"
"The tests will be graded normally"
Generally, the timeframe of the answers (does the answer need to be something that happens this year, or can it happen for next years test). This could have been rectified if I had more clear resolution criteria in the beginning and did unlinked multiple choice, but unfortunately I did not think about this.
@cc6 I still think that an N/A isn't the right thing to do. You can see my lower comment for an explanation, but the key points were:
#1: Why "nothing" isn't ambiguous:
"MAA isn't doing nothing. More specifically, MAA has done the following:
Permanently disqualified the testing center from ever hosting the AMCs again,
Created an anti-cheating algorithm to detect cheating testing centers/students (see the official MAA announcement),
Placed additional heavy restrictions on 2023 AMC 12A scores from China, more information can be found here.
There was also an "Other" option on this market the whole time (which branched off into the current leading answers). Furthermore, the phrase "do nothing" is quite easy to interpret at face value: the situation where MAA does nothing regarding the leak. Analogously, the "Other" option naturally encompasses all other possibilities, including the case where MAA takes at least some significant nontrivial action (which is the case here). Interpreting the options at face value isn't hard, furthermore, you always had the choice to clarify the meanings of the answers in the comment section."
#2: Why to not N/A:
"The fact that "Other" was undervalued by people and "Nothing" was overvalued by people is a result of people making poor choices with their bets. You overpredicted the probability of the market resolving to "Nothing", you didn't ask for clarification, and so you will pay the price. This, fundamentally, is capitalism. You win some and you lose some, and when you lose some, you don't ask for your money back. That would be unfair and selfish toward the people who realized that "Nothing" was overvalued and bet analogously. This market should NOT resolve to N/A, especially when considering that a considerable number of the people who profited from the above already withdrew their mana/money (in addition to the above)."
That being said, there's definitely a way to resolve this that isn't N/A. Here's what you can do:
(1) Reopen this market and keep it open for another ~1 year to see the long-term changes (so you can resolve the less ambiguous choices),
(2) Resolve each option percentage-wise based on how "accurate" it is. This isn't hard, and you can ask the comment section if you need help deciding the percentage values. More specifically, you resolve the obviously wrong choices to 0% and then resolve all the "partially right" choices to some probability strictly between 0% and 100%.
This works because this allows people to bet "normally" on the answer choices that probably will happen and that are reasonable (like "Disqualify competitors suspected of cheating"), but betting on more "high-risk" choices (like "Restructure their procedures for distributing the competitions" and "AMC 10/12A doesn't count for USA(J)MO index") also works. More specifically, high-risk choices like the above will get a very big chunk of the probability if they are true (for the latter choice, perhaps even all of it), but will get 0% if they are false. On the other hand, lower-risk choices have smaller variations in resolution percentages, with them (most likely) resolving higher if there are no high-risk options that turn out true. In other words, it's still possible to continue the market as-is such that there's no confusion. (The current market probabilities are relatively consistent with the probabilities if the system were to be implemented, and I can guess that a significant proportion of recent traders on this market bet while roughly assuming this system.)
Some may argue that this method of resolution is too subjective to be used effectively. Although I do understand this logic, this can be solved through a combination of common sense and asking the comments section (if necessary). For example, if the scores suddenly don't count for USA(J)MO index, then you can resolve that at somewhere from 90-100% (even 100% would be valid), and otherwise resolve that to 0%. On the other hand, if nothing big happens and there are 5 "reasonably correct" answer choices, then you can resolve them in the ballpark of 20%, with arguments for fine-tuning each number provided by debates (if necessary) in the comments section.
In summary, virtually all of the ambiguity can be resolved by using this system, as making a compromise between two sides with percentages is substantially easier than with a binary yes/no system. This market can keep going, and ultimately resolve, via this system. If you have any questions, or any proposed modifications, feel free to ask and I will be happy to answer them. N/A resolutions are only meant to be used when there's no other choice, and that's far from the case here. Markets can be reopened and/or reresolved, and if you need help with this, feel free to ask a mod or admin in the Manifold discord server. For both these reasons and in the reasons in my previous post, this market should not be N/A'ed, and the above provides a clear way of doing so.
@Math4Life2020 @cc6 I second this. Considering how many other options (including DQing the testing center, etc.) have actually happened, I believe that this market should most certainly not resolve to "N/A." This is pretty unfair to the people who got in and got out of the market, profiting off of market forces. A better way would be to partially resolve each option that did happen.
@AdvaithA @Math4Life2020 I did some additional research, and the suspension of the testing centers and change in how scoring works was not done by the MAA, but a Chinese company that distributes tests there. It's unclear whether they were told to take action by the MAA or did it independently. It could be a combination of both. The resolution of those answers is now unclear.
@cc6 “Nothing” has been unambiguously not YES for the last month, since this comment:
“I am not going to resolve it to Nothing though, either way“
@AdvaithA @Math4Life2020 after talking to some moderators and other users on discord, I decided to resolve partially to all correct options
@cc6 Yeah this market is hard to resolve. You're probably going to have to do N/A :(. Although may I say nothing seems right...
@cc6 Highly doubt they would cancel AIME qual for it though because they didn't tell people in advance
@cc6 💀 touch grass, but also does the statement that they posted about disqualifying the testing center also count as disqualifying competition managers that leaked the test? cuz technically they disqualified the site