MANIFOLD
Will simple-bench scores be reported in a major AI lab paper or blog post by the end of 2024?
6
Ṁ1kṀ770
Jan 1
29%
chance

Simple-bench.com , from the YouTube channel AI Explained, asks LLMs very simple multiple choice world modeling questions. They can resemble logic puzzles but usually have an obvious gotcha. However, even when informing the LLMs about this tricky nature, most do poorly. No LLM has cracked 30%, but humans average over 90%.

The questions may undergo revision but still resolves to Yes if it is a different version of the initial benchmark by the same author.

Market context
AI
LLMs
Get
Ṁ1,000
to start trading!

People are also trading

In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?
1/22/32
Which AI lab will have the highest Sudoku-Bench score by March 1, 2026?
What will AI score on TheAgentCompany benchmark in early 2026?
46% chance
In what year will AI achieve a score of 95% or higher on the PhysBench leaderboard?
2036
In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?
10/29/27
When will an AI lab report 50% or better result on ZeroBench?
In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?
4/6/28
Will an AI co-author a mathematics research paper published in a reputable journal before the end of 2026?
36% chance
Will AI top level capabilities generally be judged by question and answer benchmarks in 2029?
29% chance
In what year will AI achieve a score of 95% or higher on the LiveCodeBench leaderboard?
3/21/28
© Manifold Markets, Inc.•Terms•Privacy