
What will be the human baseline for the Abstraction and Reasoning Corpus (ARC-AGI)?
18
1kṀ3065resolved Sep 10
Resolved
NO90-100%
Resolved
NO75-90%
Resolved
YES60-75%
Resolved
NO35-60%
Resolved
NO15-35%
Resolved
NO0-15%
There is not currently an established human baseline for François Chollet's Abstraction and Reasoning Corpus (ARC-AGI).
On the public training set humans solve 84% of the tasks.
It it known that the public training set is easier than the public evaluation set and the private evaluation set. The public and private evaluation sets are apparently the roughly the same level of difficulty.
For the first credible human baseline study, what fraction of evaluation set tasks will humans successfully solve?
Note that this can include tasks from either the public or private evaluation sets.
In the extremely unlikely case that the number would fit in two intervals, the lowest will be chosen.
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
🏅 Top traders
# | Name | Total profit |
---|---|---|
1 | Ṁ240 | |
2 | Ṁ106 | |
3 | Ṁ106 | |
4 | Ṁ80 | |
5 | Ṁ35 |
People are also trading
Related questions
AI solves the Abstraction and Reasoning Corpus (ARC) by 2028
34% chance
Will the ARC Prize Foundation succeed at making a new benchmark that is easy for humans but still hard for the best AIs?
88% chance
Will ARC's Heuristic Arguments research substantially advance AI alignment before 2027?
26% chance
By when will we have AGI?
Will we reach "weak AGI" by the end of 2025?
24% chance
Will we have an AGI as smart as a "generally educated human" by the end of 2025?
22% chance
Will we have strong AGI by 2030? (metaculus criteria)
35% chance
ARC-AGI-2 Top Score >=50% in 2025?
19% chance
By when will 85% be reached on the public evaluation set on ARC-AGI-Pub?
When will there be an AGI for most tasks I care about?