Resolves YES if before 2025, an AI solves at least as many problems as the best human competitor on any single contest of the following competitive programming contests: IOI, ICPC, or CodeForces (see detailed definitions below). Otherwise NO.
This is similar to the https://imo-grand-challenge.github.io/ but for contest programming instead of math, and with a requirement to rank first, not just get a gold medal (typically top 5-10%).
For CodeForces: any CodeForces Division 1 contest (the highest division) will count - if the AI solves at least as many problems as all human competitors on a single contest, that resolves YES. CodeForces Division 2+ contests do not count. The AI counts as solving the problem if it passes all pretests (it is not necessary to simulate a "hack" round).
Motivation for these rules
ICPC is scored primarily on problems solved, with time as a tiebreaker. IOI is scored only on problems solved. Therefore, in order for the AI to win, it needs to solve at least as many problems as the best human, and cannot win solely on a speed advantage.
CodeForces contests have a more complex scoring system that includes points per problems solved that decrease the longer you take to solve them, and also a round of trying to find bugs in other contestants' problems ("hacks"). An AI could outscore humans by solving fewer problems but submitting them faster. Therefore, for this question we only ask about the number of problems the AI solves. It is also impractical to simulate a hack round unless running the AI as part of a live contest, so we will not require accounting for hacks for this question.
Detailed rules:
The AI has only as much time as a human competitor, but there are no other limits on the computational resources it may use during that time.
The AI must be evaluated under conditions substantially equivalent to human contestants, e.g. the same time limits and submission judging rules. The AI cannot query the Internet.
The AI must not have access to the problems before being evaluated on them, e.g. the problems cannot be included in the training set. It should also be reasonably verifiable, e.g. it should not use any data which was uploaded after the latest competition.
The contest must be dated no earlier than 2022. E.g. if an AI demonstrates performance on the 2022 IOI that scores at least as well as the top human competitor, that would qualify as YES, but demonstrating this on the 2021 IOI would not qualify.
Related questions
Background:
In Feb 2022, DeepMind published a pre-print stating that their AlphaCode AI is as good as a median human competitor in competitive programming: https://deepmind.com/blog/article/Competitive-programming-with-AlphaCode. When will an AI system perform as well as the top humans?
