When will no-calculator SOTA on the MATH dataset pass 90%?
10
Ṁ660Ṁ1.2kresolved Nov 21
100%92%
Before 2025
0.2%Other
7%
2025-2027
0.3%
2027-2029
0.0%
2029-2031
0.0%
2031-2033
0.0%
2033-2035
0.0%
2035-2040
0.0%
2040-never
This question is managed and resolved by Manifold.
Market context
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Trader | Total profit |
|---|---|---|
| 1 | Ṁ58 | |
| 2 | Ṁ47 | |
| 3 | Ṁ47 | |
| 4 | Ṁ35 | |
| 5 | Ṁ9 |
Sort by:
Math 91.6 released and publicly available: https://x.com/deepseek_ai/status/1859200145037869485?t=ljaJVfZgjJjTSKjRP6vf4g&s=19
GPT-4 is at 42.5% pass@1 https://arxiv.org/pdf/2303.12712.pdf#page=36. It seems that it is possible to use majority voting (the same thing Minerva did). I would expect such a model to be around 65%.
People are also trading
Related questions
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
87% chance
When will a single model first achieve 10@k solve rate >= 90% on the CodeContests dataset?
When will SOTA for Atari 100k pass human median and mean score on all 57 games?
BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
86% chance
[Carlini questions] SOTA AI scores better than X% of other participants in competitive programming contest by 2027
91.5
[Carlini questions] SOTA AI scores better than X% of other participants in competitive programming contest by 2030
95.2