Before 2026, will an AI system be publicly demonstrated or released, which can determine with high reliability whether a touch move violation occurred in videos of chess games? The system should point out ambiguous situations and provide reasonable evaluations for edge cases.

To meet the bar for "high reliability," red teamers (and/or the public) must have been given ample opportunity to search for videos of real chess games which the system misevaluates. Though the system does not need to be perfect, a YES resolution requires evaluators (or broad public consensus) to agree that the system decides whether a touch move violation has occurred with comparable reliability to a careful human who is knowledgeable about the rules. The system should very rarely miss blatant violations (<1%). In cases that humans consider ambiguous, the system should generally (in >90% of cases) provide reasonable justifications for its decision.

This market gauges the reliability of video-capable multimodal AI systems in detecting semantically sensitive information.