
Short-term AI 3.3: By June 2024 will SOTA on HumanEval be >= 99%?
10
190Ṁ685resolved Jun 5
Resolved
NO1H
6H
1D
1W
1M
ALL
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
🏅 Top traders
# | Name | Total profit |
---|---|---|
1 | Ṁ40 | |
2 | Ṁ14 | |
3 | Ṁ9 | |
4 | Ṁ7 | |
5 | Ṁ5 |
People are also trading
In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?
12/5/27
What will be true of the SOTA AI on the FrontierMath benchmark, before 2026?
Will AI solve 100% of solvable MTurk problems by July 2028?
32% chance
What will be the best AI performance on Humanity's Last Exam by December 31st 2025?
BIG-bench accuracy 75% #3: Will SOTA for a single model on BIG-bench pass 75% by the start of 2026?
86% chance
In what year will AI achieve a score of 95% or higher on the PhysBench leaderboard?
2036
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
Will humans create a SOTA AI model without Multi-Layer Perceptrons by 2029?
39% chance
Will AI pass the Longbets version of the Turing test by the end of 2029?
53% chance
Sort by:
@PlasmaBallin I'm just following the links in the description. Which should be straightforward (can post pics) but it's possible the linked source could be missing models? (i see one comment noting another). but maybe it's easiest just to go on the linked source
@thooton I think it's quite plausible that the test set will end up in the training set in some hard to detect way. I will exclude models for this if it's known their training set is poisoned (I assume Papers With Code would exclude them as well), but for most large language models the pre-training data is not public.
People are also trading
Related questions
In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?
12/5/27
What will be true of the SOTA AI on the FrontierMath benchmark, before 2026?
Will AI solve 100% of solvable MTurk problems by July 2028?
32% chance
What will be the best AI performance on Humanity's Last Exam by December 31st 2025?
BIG-bench accuracy 75% #3: Will SOTA for a single model on BIG-bench pass 75% by the start of 2026?
86% chance
In what year will AI achieve a score of 95% or higher on the PhysBench leaderboard?
2036
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
Will humans create a SOTA AI model without Multi-Layer Perceptrons by 2029?
39% chance
Will AI pass the Longbets version of the Turing test by the end of 2029?
53% chance