BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
7
150Ṁ307
2028
87%
chance

  • Benchmark

  • Only the sub benchmarks that are scored as an accuracy (i.e. from 0-100%) will be included (I think that's all of them but I'm not sure)

  • It must be a single model. If Model A achieves 75% on half and Model B achieves 75% on the other half that does not resolve the question YES

  • Ensemble models are fine but something like "run Model A on this benchmark and model B on this other benchmark" is not. If there is model selection is must be learned and it cannot include the current benchmark as an input.

AI
Technical AI Timelines
Get
Ṁ1,000
to start trading!


People are also trading

BIG-bench accuracy 75% #3: Will SOTA for a single model on BIG-bench pass 75% by the start of 2026?
86% chance
BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
86% chance
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
6% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
61% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2026?
What will be the highest score achieved on SWE-Bench Verified in 2025?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
© Manifold Markets, Inc.•Terms•Privacy