What will be the human baseline for the Abstraction and Reasoning Corpus (ARC-AGI)?
Basic
16
3.0k
2027
4%
0-15%
5%
15-35%
27%
35-60%
25%
60-75%
80%
75-90%
7%
90-100%

There is not currently an established human baseline for François Chollet's Abstraction and Reasoning Corpus (ARC-AGI).

On the public training set humans solve 84% of the tasks.

It it known that the public training set is easier than the public evaluation set and the private evaluation set. The public and private evaluation sets are apparently the roughly the same level of difficulty.

For the first credible human baseline study, what fraction of evaluation set tasks will humans successfully solve?

Note that this can include tasks from either the public or private evaluation sets.

In the extremely unlikely case that the number would fit in two intervals, the lowest will be chosen.

Get Ṁ600 play money
Sort by:
bought Ṁ100 35-60% NO

Only one interval resolves YES, right? Shouldn't this have been a linked market?

Oh yeah… hmm I think I messed up