ARC-AGI Prize 2024 (Dwarkesh Podcast) - Which of these scores will be achieved during the 2024 competition?
AGI TimelinesAITechnical AI TimelinesAGIAI Benchmarks
➕
Plus
62
Ṁ73k
Dec 4
81%
≥50%
39%
≥70%
14%
≥85%

Dwarkesh Patel's podcast on June 11, 2024 had guests Francois Chollet and Mike Knoop launching the $1M ARC-AGI Prize.

https://www.dwarkeshpatel.com/p/francois-chollet

https://arcprize.org/

This market resolves independently as "YES" or "NO" for each of the three thresholds based on whether they are achieved during the 2024 competition.

ARC Prize measures AGI progress using the ARC-AGI private evaluation set, the leaderboard is here. Validity of scores will be based on the contest rules and judgement of the Arc Prize sponsors.

https://www.kaggle.com/competitions/arc-prize-2024/rules

Get Ṁ600 play money

Related questions

7 Comments
56 Holders
278 Trades
Sort by:
reposted

I’m surprised “>50%” is only at 64% right now. Someone’s already managed to get 50% on ARC-AGI with GPT-4o using specific prompts, and it probably won’t be that hard to replicate.

The result on the leaderboard is 42%/43%

Yea because the admins haven’t managed to replicate it yet, but they’ll probably hear the news and replicate it soon

Sorry are you talking about https://www.lesswrong.com/posts/Rdwui3wHxCeKb7feK/getting-50-sota-on-arc-agi-with-gpt-4o or something else? The former is the first entry on arc AGI pub at https://arcprize.org/leaderboard

bought Ṁ500 ≥70% YES

https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt
ineligible for the prize though

reposted

neat benchmark,

IMO not that hard to solve, just have the synthetic data for it trained in just like ChessGPT or anything alike

They wouldn't count that.

Related questions