When will SOTA on MMLU STEM be >= 90%?
Basic
8
Ṁ584resolved Oct 4
1D
1W
1M
ALL
100%85%
2024-2025
0.1%
Before 2023
1.4%
2023-2024
9%
2025-2026
3%
2026-2027
0.2%
2027-2032
0.8%
2032-never
Get
1,000
and1.00
Sort by:
@NcyRocks Sorry: python style ranges, so X - Y means after the start of X and before the start of Y. If I can I'll edit the options to make that clearer
Related questions
Related questions
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?
12% chance
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
16% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
12% chance
By 2026, will it be standard practice to sandbox SOTA LLMs?
29% chance
BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?
65% chance
When will no-calculator SOTA on the MATH dataset pass 90%?
Will there be a period of 12 contiguous months during which no new compute-SOTA LM is released, by Jan 1, 2033?
71% chance
Will the transformer architecture be replaced in SOTA LLMs by 2028?
66% chance
When will SOTA for Atari 100k pass human median and mean score on all 57 games?