When will a single model first achieve 10@k solve rate >= 90% on the CodeContests dataset?
8
820Ṁ3822032
7%
1%
Before 2023
4%
2023-2023
19%
2024-2025
27%
2025-2026
20%
2026-2027
19%
2027-2032
3%
2032-never
10@k - unlimited samples may be generated, but at most 10 code samples may be submitted per problem.
I will post my own responses and pick whichever one of those turns out to be correct.
CodeContests dataset: https://github.com/deepmind/code_contests
Close date updated to 2050-01-01 11:59 pm
Apr 8, 4:16pm: Options are written as [a, b), so 2024-2025 means occurring after the start of 2024 and before the start of 2025.
Close date updated to 2032-01-01 11:59 pm
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
People are also trading
Related questions
Will any AI model achieve > 40% on Frontier Math before 2026?
68% chance
Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?
59% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
42% chance
Top score on codeforces by an AI model at the end of 2025
Will GPT-5 reach a 1000 rating on Codeforces?
96% chance
By when will SAT solvers be 10x faster (than 2023 on same hardware)?
Will an AI model achieve superhuman ELO on Codeforces by the 31 December 2025?
35% chance
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
10% chance
Will OpenAI models achieve ≥90% on SimpleBench by the end of 2025?
37% chance
When Will 9x9 Go Be Solved?