Will a text model achieve 100% performance on the MMLU in five years?
3
42
Ṁ41Ṁ90
2028
28%
chance
1D
1W
1M
ALL
This video predicts that a model will achieve 100% performance on the Massive Multitask Language Understanding Benchmark.
This will resolve YES if a model is listed as having achieved 100 percent performance on the MMLU on papers with code, in a technical document, or published article by 3:00 PM EST June 28, 2028. This will resolve NO if a model is not listed as ahving achieved 100 percent performance on the MMLU.
Get Ṁ200 play money
Related questions
Related questions
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
67% chance
Will real-time text-to-video generation be viable by 2030?
66% chance
Will a single model achieve superhuman performance on all OpenAI gym environments by 2025?
39% chance
By 2027, will it be generally agreed upon that LLM produced text > human text for training LLMs?
63% chance
Will language models be able to solve simple graphical mazes by the end of 2025?
65% chance
Will any open-source model achieve GPT-4 level performance on MMLU through 2024?
83% chance
Will a single model achieve superhuman performance on all Atari environments by 2025?
40% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
Will a GPT-5-worthy model be able to break 50% on NYT Connections benchmark?
69% chance
Will an open model with comparable results to GPT-4 on MMLU be released by the end of July 2024?
75% chance