When will a single model first achieve 10@k solve rate >= 90% on the CodeContests dataset?
7
820Ṁ3572032
7%
1%
Before 2023
4%
2023-2023
19%
2024-2025
27%
2025-2026
20%
2026-2027
19%
2027-2032
3%
2032-never
10@k - unlimited samples may be generated, but at most 10 code samples may be submitted per problem.
I will post my own responses and pick whichever one of those turns out to be correct.
CodeContests dataset: https://github.com/deepmind/code_contests
Close date updated to 2050-01-01 11:59 pm
Apr 8, 4:16pm: Options are written as [a, b), so 2024-2025 means occurring after the start of 2024 and before the start of 2025.
Close date updated to 2032-01-01 11:59 pm
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
1,000Sort by:
@ValeryCherepanov Yeah the before 2023 option seems like free money to me, I think the market is just old and people don't have it on their radar.
Related questions
Related questions
Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?
72% chance
Top score on codeforces by an AI model at the end of 2025
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
18% chance
Will GPT-5 reach a 1000 rating on Codeforces?
96% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
45% chance
By when will SAT solvers be 10x faster (than 2023 on same hardware)?
When will an AI model be better than me at competitive programming?
Will an AI model achieve superhuman ELO on Codeforces by the 31 December 2025?
68% chance
Will OpenAI models achieve ≥90% on SimpleBench by the end of 2025?
42% chance
When Will 9x9 Go Be Solved?