A few days ago, Twitter user Liron Shapira posted the following challenge:

To which I replied:

Will I be correct in predicting that AI will not have this capability within 12 months? This market resolves YES if a bot of any kind is shown to have completed The Legend of Zelda for the NES within 12 months of the date of the original Tweet (which will be September 21, 2024), and NO otherwise.

Considerations:

• The playthrough must begin at console boot, and is considered to have completed the game upon touching Princess Zelda in Level 9.

• Any official release of the game is okay.

• Since this was intended as a test of the bot's "reasoning" and "planning" abilities, the bot should spend at least 5 minutes actually playing the game. This is meant to prevent it from using any ACE-like techniques that allow it to skip most of the game without having to reason or plan. (Although if it discovers a method of completing the game in under 5 minutes, I will be duly impressed nonetheless.)

• Although the original Tweet suggested that LLMs should be involved in some way, I won't worry about that. It doesn't matter if the bot makes use of LLM techniques, or neural networks, or even machine learning. If someone comes up with a purely symbolic script capable of completing Z1, I'll happily resolve that as YES (and count it as a victory for symbolic algorithms over ML). The only requirement is that it shouldn't have any pre-existing knowledge about the game hardcoded, either in its actual code or in the weights of any neural network it makes use of. (This includes being trained on gameplay recordings of the game.)

• It's fine if the bot "reads" the game manual before starting its playthrough, in whatever format the developers choose to present it.

• It's fine if the bot is trained on gameplay recordings of other games, as long as it has not been exposed to Z1 specifically.

• It doesn't matter what the bot's success rate is, as long as it has been documented to complete the game at least once before the close date.

• It doesn't matter how long it takes the bot to complete the game, as long as it does so at least once before the close date.

• [Added 2023/09/24] See my long post in the comments for some discussion of what "prior knowledge" entails. Basically, isolated factoids about puzzles in the game are fine; fine-grained knowledge about the shape of the map is not. If, for some reason, this bot requires an LLM, and all existing general-purpose LLMs of sufficient capabilities already have the game memorized at a fine-grained level, it's acceptable to run this test on ROM hacks or something that they do not have memorized.

• I will not be betting in this market.