49. Will AI win a programming competition in 2023?
resolved Jan 1

This will resolve positively if a major news source reports that an AI entered a programming competition with at least two other good participants, and won the top prize. A good participant will be defined as someone who could be expected to perform above the level of an average big tech company employee; if there are at least twenty-five participants not specifically selected against being skilled, this will be considered true by default. The competition could be sponsored by the AI company as long as it meets the other criteria and is generally considered fair.

This is question #49 in the Astral Codex Ten 2023 Prediction Contest. The contest rules and full list of questions are available here. Market will resolve according to Scott Alexander’s judgment, as given through future posts on Astral Codex Ten.

Get Ṁ600 play money

🏅 Top traders

#NameTotal profit
Sort by:

Can't find any evidence for YES. Resolves NO

predicted NO

@oh Good enough for me. Thanks!

@VictorLevoso some news we missed?

sold Ṁ10 of YES

Oh, I thought this meant someone using AI as part of their entry into a programming contest, not the AI itself submitting its own work. That's less likely.

bought Ṁ200 of YES

I said 80% for this one in my predictions: https://www.newslettr.com/p/more-on-prediction-markets

The main concern I have is "there aren't that many programming contests".

This has not been rising at Metaculus, although Manifold has been catching up to it. This should damn well be rising, as both progress has been impressive recently with no signs of actual stopping due to risk, and there is now a ton more hype and funding and reason to put on this kind of show, and more incentive for ‘a major news outlet to report it,’ note that this does not require a front page or anything like that. I’m going to put M100 on YES, bringing this to 42%. I notice my gut-level fair is at least 50% here.

- Zvi Mowshowitz

bought Ṁ200 of YES

A promising new paper on getting LLMs to solve coding problems:


Note that the question defines: "A good participant will be defined as someone who could be expected to perform above the level of an average big tech company employee".

Now, big tech interviews do not particularly select for competitive programming performance, nor do competitive programmers select for big tech employment that much.

@dp The thing about 25 participants automatically qualifying the contest points even more to YES. However, a contest like this actually being held in 2023 and being notable enough for reporting is... not that likely.

More related questions