Advent of Code is an annual set of Christmas-themed computer programming challenges that follow an Advent calendar.[1][2] It has been running since 2015.[3]
The programming puzzles cover a variety of skill sets and skill levels and can be solved using any programming language. Participants also compete based on speed on both global and private leaderboards.[1]
This question resolves to YES if and only if someone legibly claims to have gotten a top 10 placement on the two-star leaderboard for any of the 24 first days of Advent of Code 2024 with a solution either produced directly from the statement by an LLM, or with code entirely written by an AI code assistant.
I will exclude days where the quickest 2-star solve was faster than 3 minutes as unrepresentative and too noisy.
This question closes at EOY 2024.
1,000
3.00This question is very badly written. LLMs are not "stumped" if they take too long to solve a problem, they're stumped if the output does not work.
The constraints give so far basically make it impossible for any YES resolution, as any day where AI does well is automatically excluded. The only way is if someone gets a solution with AI then all AI solvers intentionally sandbag. Or just one LLM cracks it after a bunch of prompt engineering but all other AIs fail.
It would be much simpler to calibrate difficulty with a different unbiased metric, like time taken by 100th solve. Or total number of solves after 24h.
@vlads potentially yes. The goal here is to try to only consider problems of a certain minimal level of complexity
@0x413124 So far — ten days in — the problems in 2024 don't seem to reach enough complexity to stump an LLM solution for long. For comparison, here are the quickest solves for the first ten days in 2023 and 2024, listed in order of increasing time taken:
2023: 1m22s 1m34s 1m48s 2m05s 2m24s 4m24s 5m09s 8m38s 8m45s 10m04s
2024: 9s 38s 49s 55s 1m01s 1m08s 1m10s 1m11s 1m42s 1m57s