Seems hard to test. Do you have a programming team in mind? Supposing they don't follow these rules, (ie., randomly selected games, days not years for connecting, practice/play time, and somehow evident or reported to you/us) would you resolve N/A, or draw your best conclusions?

"AI may learn with ease,
But can it match human expertise?
In gaming, it's a whole new fight,
But with time, perhaps it'll play right."

Pretty sure this already occurred

@Gigacasting deepmind was able to do this with atari games

so this resolving to yes is just a matter of incremental progress. We'd have to suffer another winter imo for this to resolve to no.

To resolve positively, is it required that the AI can complete training using only the GPU on a normal person's computer? This requirement seems implied by the original article's use of the phrase "off-the-shelf AI" but I could be misinterpreting what that means.

@MichaelDickens Alternatively, "off-the-shelf" just means it's a general-purpose game-playing AI, rather than custom-built for a specific game

This seems AGI-complete, in that it's isomorphic to saying "an ai will be able to perform totally arbitrary tasks on a computer's ui which involve complex planning (basically the whole strategy genre)"

@AaronKaufman I don't think so. The prompt is giving it a few days to connect and practice, and a random game on average has only a few inputs you can perform, as well as several limiting rules. If the random game is Minecraft the AI might be screwed, but if it's Celeste maybe it is getting close already?

@AaronKaufman It does involve re-training of weights. How much can the model learn with the given set of weights such that it is not only able to perform well at the tasks its optimized its weights for but also learn to do new tasks from very very few examples

80%