Will there be a program achieving at least 50% accuracy on the Abstraction and Reasoning Corpus by the end of 2023?
6
109
Ṁ230Ṁ150
resolved Jan 15
Resolved
NO1D
1W
1M
ALL
https://github.com/fchollet/ARC
Get Ṁ200 play money
Related questions
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ25 | |
| 2 | Ṁ11 | |
| 3 | Ṁ9 | |
| 4 | Ṁ8 | |
| 5 | Ṁ8 |
Sort by:
50% of what? There are 800 questions available in the repository (400 training; 400 evaluation), and you could trivially write a program that is >50% accurate just by hardcoding the solutions to those. (Probably contrary to the spirit of this market, but technically valid.) The ARCathon competition tests entrants on an additional 100 unpublished problems.
Related questions
Will an AI agent system be able to score at least 40% on level 3 tasks in the GAIA benchmark before 2025.
49% chance
AI solves the Abstraction and Reasoning Corpus (ARC) by 2028
33% chance
By the end of 2026, will we have transparency into any useful internal pattern within a Large Language Model whose semantics would have been unfamiliar to AI and cognitive science in 2006?
53% chance
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
67% chance
Will >50% of the tasks in the WebArena benchmark be solved by EOY 2024?
62% chance
Will an AI be capable of achieving a perfect score on the Putnam exam before 2030?
33% chance
Will any AI be able to explain formal language proofs to >=50% of IMO problems by the start of 2025?
63% chance
Will an AI be able to convert recent mathematical results into a fully formal proofs that can be verified by a mainstream proof assistant by 2025?
23% chance
Will any AI be able to formalize >=90% of IMO problems by the start of 2025?
32% chance
In 2029, will any AI be able to construct "reasonably" bug-free code of >= 10k LOC from a natural language specification? (Gary Marcus benchmark #4)
76% chance