Will an LLM report >50% score on ARC in 2025?
19
1kṀ3984
Dec 31
85%
chance

ARC is a general-purpose AI eval designed to test intelligence as opposed to memorization. https://arcprize.org/arc

This market resolves to Yes if there are public demonstrations of general-purpose base models from OpenAI, Anthropic or Deepmind that solve at least 50% of the ARC questions before 2026.

(note that this is separate from winning the arc price, which requires only using open-source models)

AI
Technical AI Timelines
Get
Ṁ1,000
to start trading!
Sort by:

Current leaderboard:

I guess you're not counting reasoning models like o1 or o3? Are you also ignoring the compute limit?

bought Ṁ1,000 YES

I assume this resolves YES if it happens in 2024?

Yes!

People are also trading

Will an LLM get > 50% on hard problems on LiveCodeBench Pro?
48% chance
LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?
60% chance
Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?
67% chance
Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?
50% chance
Will China have the best LLM by the end of 2025?
17% chance
Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?
66% chance
Will LLMs mostly overcome the Reversal Curse by the end of 2025?
72% chance
What will be true of Anthropic's best LLM by EOY 2025?
ARC-AGI-2 Top Score >=50% in 2025?
21% chance
Will RL work for LLMs "spill over" to the rest of RL by 2026?
34% chance
© Manifold Markets, Inc.•Terms•Privacy