To enable a more accurate assessment of AI agents in challenging exploratory environments, we introduce TextQuests, a benchmark based on the Infocom suite of interactive fiction games. These text-based adventures, which can take human players over 30 hours and require hundreds of precise actions to solve, serve as an effective proxy for evaluating AI agents on focused, stateful tasks. The benchmark is specifically designed to assess an LLM agent's capacity for self-contained problem-solving by precluding the use of external tools, thereby focusing on intrinsic long-context reasoning capabilities in an exploratory environment characterized by the need for trial-and-error learning and sustained problem-solving within a single interactive session.
TextQuests consists of 25 classic interactive fiction games of varying difficulty developed by Infocom. These games were the preeminent text-based adventures of the 1980s, challenging players to interact with story-rich worlds using natural language commands.
In what year will AI achieve 90% of progress in TextQuests benchmark? This market is about the results with no clues.
1,000