Will OpenAI's next foundation model score at least 75% in MMMU?
Basic
4
Ṁ86resolved Sep 16
Resolved
YES1D
1W
1M
ALL
This question is about the MMMU benchmark, not the MMLU benchmark where many models already achieve over 75% accuracy.
Resolve this to YES if OpenAI's next generation foundation model* achieves an average score of at least 75% on the MMMU benchmark.
While prompt engineering could potentially boost performance after the release, I'm not sure how long it might take to realize those gains. So rather than waiting, I'll simply resolve this question once OpenAI releases their next foundation model iteration.
*GPT-4o doesn't count; the model must not be included in the GPT-4 family by OpenAI.
Related Questions:
https://manifold.markets/3684/will-openais-nextgeneration-models
https://manifold.markets/3684/will-openais-nextgen-mathfocused-mo
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Sort by:
Related questions
Related questions
Will an AI score over 80% on FrontierMath Benchmark in 2025
23% chance
Will an AI system be reported by OpenAI as of December 31st 2025 as having a pre-mitigation score of...
Will OpenAI models achieve ≥90% on SimpleBench by the end of 2025?
46% chance
What will OpenAI's next foundation model be called?
When will OpenAI release a more capable LLM?
Will OpenAI's next major LLM (after GPT-4) surpass 70% accuracy on the GPQA benchmark?
66% chance
What year will the first AI exceed 80% on MLE-bench?
Will OpenAI's next major LLM (after GPT-4) achieve over 50% resolution rate on the SWE-bench benchmark?
80% chance
Will OpenAI's next major LLM (after GPT-4) surpass 74% accuracy on the GPQA benchmark?
85% chance
Will the top model by OpenAI rank 3rd (or lower) behind 2 other model families at any point before 2026?
41% chance