By when will 85% be reached on the public evaluation set on ARC-AGI-Pub? | Manifold

By when will 85% be reached on the public evaluation set on ARC-AGI-Pub?

66

10kṀ57k

2030

1H

6H

1D

1W

1M

ALL

82%

01/01/2026

92%

01/01/2028

95%

01/01/2030

Resolved

NO

01/01/2025

See https://arcprize.org/arc-agi-pub.

This allows for arbitrary API usage unlike the ARC-AGI prize.

I will resolve based on when the date when the method is first reported even if it is not submitted until substantially later (due to e.g. no public API).

I will apply a restriction roughly in line with the $10,000 runtime cost contraint, though it is unclear how this should resolve with respect to private models with no known API price.

I will exclude solutions that seem heavily overfit based on the semi-private evaluation set.

(Cut off data is resolved based on midnight PT.)

Technical AI Timelines

Get

1,000

to start trading!

People are also trading

Will we reach "weak AGI" by the end of 2025?

Will a major lab officially declare AGI before the end of 2025?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?

In what year will AI achieve a score of 95% or higher on the GPQA benchmark?

In what year will AI achieve a score of 95% or higher on the GSO benchmark?

In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?

Will we get AGI before August 15th 2027?

Will we get AGI before September 15th 2027?

Sort by:

@RyanGreenblatt I think this resolves NO? o3 did pass 85%, but it was well above the cost constraints.

bought Ṁ45 YES

https://arcprize.org/blog/oai-o3-pub-breakthrough

sold Ṁ477 YES

wow, 83% on public with high efficiency. So close (and I feel the mana punishment for trading too fast before reading the fine print)

If 85% is reached on the main ARC-AGI prize track and there isn't a clear reason to think this wouldn't transfer to ARC-AGI-Pub, I would also resolve this market to true even if no one submits to ARC-AGI-Pub.

People are also trading

Will we reach "weak AGI" by the end of 2025?

-6% 1d15% chance

Will a major lab officially declare AGI before the end of 2025?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?

In what year will AI achieve a score of 95% or higher on the GPQA benchmark?

In what year will AI achieve a score of 95% or higher on the GSO benchmark?

In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?

Will we get AGI before August 15th 2027?

Will we get AGI before September 15th 2027?

Related questions

Will we reach "weak AGI" by the end of 2025?

Will a major lab officially declare AGI before the end of 2025?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will AI achieve a score of 85% or higher on the SimpleBench leaderboard?

In what year will AI achieve a score of 95% or higher on the GPQA benchmark?

In what year will AI achieve a score of 95% or higher on the GSO benchmark?

In what year will AI achieve a score of 95% or higher on the PutnamBench leaderboard?

Will we get AGI before August 15th 2027?

Will we get AGI before September 15th 2027?

© Manifold Markets, Inc.•Terms•Privacy