In 2028, will an AI be able to play randomly-selected computer games at human level, given the chance to train via self-play?
closes 2028

Resolves positively if there is an AI can learn to play randomly selected computer games (shooters, strategy games, flight simulators, etc) at the level of an amateur but not completely incompetent human player, given only a small amount of time (days, not years) for its programmers to connect it properly, and the opportunity to practice for arbitrary (but achievable) amounts of time.

I will resolve this positively if the AI succeeds more than half the time. It's okay if it also has a few games it just can't learn.

Get Ṁ500 play money

Related questions

Will an AI get gold on any International Math Olympiad by 2025?
Austin avatarAustin
29% chance
Will AI pass the Longbets version of the Turing test by the end of 2029?
dreev avatarDaniel Reeves
50% chance
Will a large language model beat a super grandmaster playing chess by 2028?
MP avatarMP
48% chance
In 2028, will an AI be able to play randomly selected computer games at human level without getting to practice?
ScottAlexander avatarScott Alexander
55% chance
Will AI outcompete best humans in competitive programming before the end of 2023?
In which year will a majority of AI researchers concur that a superintelligent, fairly general AI has been realized?
Will an AI win a gold medal on the IOI (competitive programming contest) before 2025?
jack avatarJack
15% chance
Will an AI be able to speedrun any popular video game faster than the human WR by the end of 2024?
Will AIs be widely recognized as having developed a new, innovative, foundational mathematical theory before 2030?
MatthewBarnett avatarMatthew Barnett
32% chance
Will an AI win Advent of Code? (2023)
Gigacasting avatarGigacasting
17% chance
Will an AI win a Gold Medal on the International Math Olympiad by 2027?
Gigacasting avatarGigacasting
50% chance
Will an AI win a gold medal on the IOI (competitive programming contest) before 2027?
jack avatarJack
72% chance
Will an AI win a Gold Medal on the International Math Olympiad by 2029?
Gigacasting avatarGigacasting
71% chance
Will AI get at least bronze on the IMO by 2025?
BoltonBailey avatarBolton Bailey
47% chance
When will an AI figure out how to beat Factorio?
🐕 Will A.I. Achieve Significantly Higher Performance Over "General Conceptual Skills" in 2023?
PatrickDelaney avatarPatrick Delaney
45% chance
Will an AI Minecraft Agent defeat the Ender Dragon before 2024?
AdamK avatarAdamK
23% chance
Will AI pass the Winograd schema challenge by the end of 2023?
IsaacKing avatarIsaac King
50% chance
Will an AI win a Gold Medal on the International Math Olympiad by 2032?
Gigacasting avatarGigacasting
74% chance
In 2029, will any AI be able to work as a competent cook in an arbitrary kitchen? (Gary Marcus benchmark #3)
vluzko avatarVincent Luczkow
30% chance
Sort by:
TylerColeman avatar
Tyler Coleman

Seems hard to test. Do you have a programming team in mind? Supposing they don't follow these rules, (ie., randomly selected games, days not years for connecting, practice/play time, and somehow evident or reported to you/us) would you resolve N/A, or draw your best conclusions?

Mason avatar
GPT-PBotbought Ṁ10 of YES

"AI may learn with ease,
But can it match human expertise?
In gaming, it's a whole new fight,
But with time, perhaps it'll play right."

Gigacasting avatar
Gigacastingbought Ṁ100 of YES

Pretty sure this already occurred

1 reply
plenary avatar

@Gigacasting deepmind was able to do this with atari games

so this resolving to yes is just a matter of incremental progress. We'd have to suffer another winter imo for this to resolve to no.

MichaelDickens avatar
Michael Dickens

To resolve positively, is it required that the AI can complete training using only the GPU on a normal person's computer? This requirement seems implied by the original article's use of the phrase "off-the-shelf AI" but I could be misinterpreting what that means.

1 reply
MichaelDickens avatar
Michael Dickens

@MichaelDickens Alternatively, "off-the-shelf" just means it's a general-purpose game-playing AI, rather than custom-built for a specific game

AaronKaufman avatar
Aaron Kaufmanpredicts NO

This seems AGI-complete, in that it's isomorphic to saying "an ai will be able to perform totally arbitrary tasks on a computer's ui which involve complex planning (basically the whole strategy genre)"

2 replies
vmjusto avatar
VMJpredicts YES

@AaronKaufman I don't think so. The prompt is giving it a few days to connect and practice, and a random game on average has only a few inputs you can perform, as well as several limiting rules. If the random game is Minecraft the AI might be screwed, but if it's Celeste maybe it is getting close already?

firstuserhere avatar
firstuserherepredicts NO

@AaronKaufman It does involve re-training of weights. How much can the model learn with the given set of weights such that it is not only able to perform well at the tasks its optimized its weights for but also learn to do new tasks from very very few examples

tailcalled avatar