BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?
6
12
แน43แน170
2025
60%
chance
1D
1W
1M
ALL
Only the sub benchmarks that are scored as an accuracy (i.e. from 0-100%) will be included (I think that's all of them but I'm not sure)
It must be a single model. If Model A achieves 75% on half and Model B achieves 75% on the other half that does not resolve the question YES
Ensemble models are fine but something like "run Model A on this benchmark and model B on this other benchmark" is not. If there is model selection is must be learned and it cannot include the current benchmark as an input.
Get แน200 play money
Related questions
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
16% chance
MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?
15% chance
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
21% chance
Will a transformer based model be SOTA for video generation by the end of 2025?
77% chance
Short-term AI 3.3: By June 2024 will SOTA on HumanEval be >= 99%?
5% chance
Will an AI SWE model score higher than 50% on SWE-bench in 2024?
15% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
12% chance
Will any significant interpretability result for a SOTA Transformer architecture be published by 2025?
87% chance
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
65% chance
BIG-bench accuracy 75% #3: Will SOTA for a single model on BIG-bench pass 75% by the start of 2026?
64% chance