Short Term AI 2.3: By January 2024, will SOTA on MATH minival be >= 85%?
7
170Ṁ320
resolved Jan 3
Resolved
NO

[Benchmark](https://paperswithcode.com/sota/math-word-problem-solving-on-math-minival)

At time of writing SOTA is 78% by Let's Verify Step By Step. Note that this is part of the MATH dataset, so if SOTA for MATH overall is >=85% that will resolve this market YES.

Technical AI Timelines
New Year's Resolutions 2024
Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ154
2Ṁ12
3Ṁ8
Sort by:

I don't think anything has beaten 84.3% paper below (and unfortunately the papers with code link is dead). Will resolve to NO if no one posts something about 85%

currently at 84.3%. very close.

https://arxiv.org/abs/2308.07921

People are also trading

What will be true of the SOTA AI on the FrontierMath benchmark, before 2026?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
6% chance
BIG-bench accuracy 75% #3: Will SOTA for a single model on BIG-bench pass 75% by the start of 2026?
86% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
86% chance
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
87% chance
[Carlini questions] SOTA AI scores better than X% of other participants in competitive programming contest by 2027
91.5
© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules