Skip to main content
MANIFOLD
A Chinese AI model will score over 85% on the MMLU before 2024
18
Ṁ350Ṁ19k
resolved Mar 16
Resolved
NO

Market context
AI
Technical AI Timelines
China
Get
Ṁ1,000
to start trading!

🏅 Top traders

#TraderTotal profit
1Ṁ366
2Ṁ42
3Ṁ8
4Ṁ5
5Ṁ3

People are also trading

Will a Chinese AI model rank in the top 10 on the LMArena Text leaderboard on August 1?
59% chance
Will any AI model score above 95% on ARC-AGI-2 by end of 2026?
90% chance
Will the top Chinese AI model exceed the top US AI model before 2028?
38% chance
The American open AI frontier will catch up to Chinese models in 2026
31% chance
Will any AI achieve a score of 25% on ARC-AGI-3 by the end of 2026?
65% chance
What year will the first AI exceed 80% on MLE-bench?
In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?
7/1/28
In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?
6/8/32
Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?
67% chance
Sort by:

Did not happen @NathanpmYoung

@qumeric The resolution date is apparently Dec 31, 2024, so I guess the creator meant "before 2024 ends"

@CDBiddulph maybe creator set resolution date incorrectly instead?

predictedYES

What do you mean by before 2024? like before 2024 jan 1?

How worried should we be about a chinese lab training on the answers, declaring they've solved MMLU, and not releasing the model so people can't tell they cheated?

People are also trading

Will a Chinese AI model rank in the top 10 on the LMArena Text leaderboard on August 1?
59% chance
Will any AI model score above 95% on ARC-AGI-2 by end of 2026?
90% chance
Will the top Chinese AI model exceed the top US AI model before 2028?
38% chance
The American open AI frontier will catch up to Chinese models in 2026
31% chance
Will any AI achieve a score of 25% on ARC-AGI-3 by the end of 2026?
65% chance
What year will the first AI exceed 80% on MLE-bench?
In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?
7/1/28
In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?
6/8/32
Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?
67% chance