
By when will 85% be reached on the public evaluation set on ARC-AGI-Pub?
65
10kṀ56k2030
1H
6H
1D
1W
1M
ALL
87%
01/01/2026
92%
01/01/2028
95%
01/01/2030
Resolved
NO01/01/2025
See https://arcprize.org/arc-agi-pub.
This allows for arbitrary API usage unlike the ARC-AGI prize.
I will resolve based on when the date when the method is first reported even if it is not submitted until substantially later (due to e.g. no public API).
I will apply a restriction roughly in line with the $10,000 runtime cost contraint, though it is unclear how this should resolve with respect to private models with no known API price.
I will exclude solutions that seem heavily overfit based on the semi-private evaluation set.
(Cut off data is resolved based on midnight PT.)
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
People are also trading
Will a major lab officially declare AGI before the end of 2025?
9% chance
Will we reach "weak AGI" by the end of 2025?
22% chance
In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?
12/13/27
In what year will AI achieve a score of 95% or higher on the GPQA benchmark?
2/25/27
In what year will AI achieve a score of 95% or higher on the GSO benchmark?
2/2/29
In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?
11/3/28
In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?
3/4/32
Will we get AGI before August 15th 2027?
14% chance
Will we get AGI before September 15th 2027?
27% chance
Sort by:
@RyanGreenblatt I think this resolves NO? o3 did pass 85%, but it was well above the cost constraints.
People are also trading
Related questions
Will a major lab officially declare AGI before the end of 2025?
9% chance
Will we reach "weak AGI" by the end of 2025?
22% chance
In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?
12/13/27
In what year will AI achieve a score of 95% or higher on the GPQA benchmark?
2/25/27
In what year will AI achieve a score of 95% or higher on the GSO benchmark?
2/2/29
In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?
11/3/28
In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?
3/4/32
Will we get AGI before August 15th 2027?
14% chance
Will we get AGI before September 15th 2027?
27% chance