Short-Term AI #2: By the end of June 2023, will SOTA on MMLU overall be >=85%?
13
Ṁ270Ṁ9.2kresolved Mar 14
Resolved
YES1H
6H
1D
1W
1M
ALL
Current SOTA is ~75%: https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu
This question is managed and resolved by Manifold.
Market context
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Trader | Total profit |
|---|---|---|
| 1 | Ṁ53 | |
| 2 | Ṁ37 | |
| 3 | Ṁ36 | |
| 4 | Ṁ20 | |
| 5 | Ṁ19 |
People are also trading
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
86% chance
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
87% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
SOTA AI at EOY 2026 a reasoning model?
94% chance
Any SOTA AI model uses human-understandable thinking medium at the end of 2028?
71% chance
Sort by:
People are also trading
Related questions
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
86% chance
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
87% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
SOTA AI at EOY 2026 a reasoning model?
94% chance
Any SOTA AI model uses human-understandable thinking medium at the end of 2028?
71% chance