By when will 85% be reached on the public evaluation set on ARC-AGI-Pub?

10kṀ94k

resolved Jan 7

ALL

Resolved

01/01/2025

Resolved

YES

01/01/2026

Resolved

YES

01/01/2028

Resolved

YES

01/01/2030

See https://arcprize.org/arc-agi-pub.

This allows for arbitrary API usage unlike the ARC-AGI prize.

I will resolve based on when the date when the method is first reported even if it is not submitted until substantially later (due to e.g. no public API).

I will apply a restriction roughly in line with the $10,000 runtime cost contraint, though it is unclear how this should resolve with respect to private models with no known API price.

I will exclude solutions that seem heavily overfit based on the semi-private evaluation set.

(Cut off data is resolved based on midnight PT.)

Market context

Technical AI Timelines

Get

1,000

to start trading!

🏅 Top traders

#	Name	Total profit
1		Ṁ1,551
2		Ṁ778
3		Ṁ776
4		Ṁ438
5		Ṁ404

People are also trading

[ACX 2026] What will be the highest score achieved on ARC-AGI-2 before 2027?

83.6

By which years will AGI be released (Hendrycks definition >95%)?

Will the ARC-AGI grand prize be claimed by end of 2026?

57% chance

In what year will AI achieve a score of 95% or higher on the GPQA benchmark?

5/25/27

In what year will AI achieve a score of 95% or higher on the GSO benchmark?

4/8/29

In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?

10/8/28

AI solves the Abstraction and Reasoning Corpus (ARC) by 2028

20% chance

In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?

2/11/32

Will we get AGI before August 15th 2027?

Sort by:

I think this market should resolved yes based on https://x.com/arcprize/status/1999182732845547795 and https://arcprize.org/leaderboard. Any objections?

@RyanGreenblatt I think this resolves NO? o3 did pass 85%, but it was well above the cost constraints.

bought Ṁ45 YES

https://arcprize.org/blog/oai-o3-pub-breakthrough

sold Ṁ477 YES

wow, 83% on public with high efficiency. So close (and I feel the mana punishment for trading too fast before reading the fine print)

If 85% is reached on the main ARC-AGI prize track and there isn't a clear reason to think this wouldn't transfer to ARC-AGI-Pub, I would also resolve this market to true even if no one submits to ARC-AGI-Pub.