Will OpenAI's next foundation model score at least 75% in MMMU?
4
100Ṁ86resolved Sep 16
Resolved
YES1H
6H
1D
1W
1M
ALL
This question is about the MMMU benchmark, not the MMLU benchmark where many models already achieve over 75% accuracy.
Resolve this to YES if OpenAI's next generation foundation model* achieves an average score of at least 75% on the MMMU benchmark.
While prompt engineering could potentially boost performance after the release, I'm not sure how long it might take to realize those gains. So rather than waiting, I'll simply resolve this question once OpenAI releases their next foundation model iteration.
*GPT-4o doesn't count; the model must not be included in the GPT-4 family by OpenAI.
Related Questions:
https://manifold.markets/3684/will-openais-nextgeneration-models
https://manifold.markets/3684/will-openais-nextgen-mathfocused-mo
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ46 | |
| 2 | Ṁ4 |
Sort by:
People are also trading
Related questions
Will OpenAI models achieve ≥90% on SimpleBench by the end of 2025?
24% chance
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
5% chance
Will OpenAI's next major LLM (after GPT-4) achieve over 50% resolution rate on the SWE-bench benchmark?
99% chance
Open-Source AI model gets perfect IMO 2026 score? [International Math Olympiad 2026]
31% chance
Will any AI model score above 95% on GRAB by the end of 2025?
40% chance
Will OpenAI's o4 get above 50% on humanity's last exam?
26% chance
When will an OpenAI model achieve a High risk level on AI Self-improvement? [metaculus]
Will the top model by OpenAI rank 3rd (or lower) behind 2 other model families at any point before 2026?
66% chance
Will an AI score over 80% on FrontierMath Benchmark in 2025
5% chance
Will a text model achieve 100% performance on the MMLU in five years?
28% chance