Background:

SimpleBench is a 200‑item multiple‑choice test designed to probe everyday human reasoning that still eludes frontier LLMs, including spatio‑temporal reasoning, social intelligence and linguistic “trick” questions. Unlike most other benchmarks, humans still outperform AI on the SimpleBench.

State of play:

• Human reference accuracy: 83.7 %

• 2025 AI accuracy (Gemini 2.5 Pro): 62.4 %

Why this milestone matters

• Everyday reasoning: Passing SimpleBench would indicate that LLMs can handle commonsense scenarios that remain brittle today.

• Benchmark head‑room: Unlike MMLU, SimpleBench has not yet been “solved”, so it is a useful yard‑stick for progress to compare AI to humans.

Resolution Criteria:

This market resolves to the year in which a fully automated AI system first achieves ≥ 85% average accuracy on SimpleBench (ALL metric), subject to all of the following:

• Verification – The claim must be confirmed by either

  1. a peer‑reviewed paper on arXiv

  2. a public leaderboard entry on SimpleBench Official Website or another credible source.

• Compute resources – Unlimited.

Fine Print:

If the resolution criteria are unsatisfied by Jan 1, 2033 the market resolves to “Not Applicable.”