Will OpenAI's next foundation model score at least 75% in MMMU?
4
Ṁ100Ṁ86resolved Sep 16
Resolved
YES1H
6H
1D
1W
1M
ALL
This question is about the MMMU benchmark, not the MMLU benchmark where many models already achieve over 75% accuracy.
Resolve this to YES if OpenAI's next generation foundation model* achieves an average score of at least 75% on the MMMU benchmark.
While prompt engineering could potentially boost performance after the release, I'm not sure how long it might take to realize those gains. So rather than waiting, I'll simply resolve this question once OpenAI releases their next foundation model iteration.
*GPT-4o doesn't count; the model must not be included in the GPT-4 family by OpenAI.
Related Questions:
https://manifold.markets/3684/will-openais-nextgeneration-models
https://manifold.markets/3684/will-openais-nextgen-mathfocused-mo
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
🏅 Top traders
| # | Trader | Total profit |
|---|---|---|
| 1 | Ṁ46 | |
| 2 | Ṁ4 |
Sort by:
People are also trading
Related questions
Will OpenAI's o4 get above 50% on humanity's last exam?
14% chance
Will any AI model score above 95% on ARC-AGI-2 by end of 2026?
91% chance
When will an AI Model first score >= 170 on the Epoch Capabilities Index?
Will a text model achieve 100% performance on the MMLU in five years?
28% chance
When will an OpenAI model achieve a High risk level on AI Self-improvement? [metaculus]
When will an OpenAI model achieve a Critical risk level on AI Self-improvement? [metaculus]