
Will o1 score ≥60% on the REBUS benchmark?
5
1kṀ1905resolved Mar 11
Resolved
YES1D
1W
1M
ALL
Update 2024-22-12 (PST): This market refers to the REBUS benchmark as described in the paper "REBUS: A Benchmark to Evaluate the Rationality of Language Models" (AI summary of creator comment)
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
🏅 Top traders
# | Name | Total profit |
---|---|---|
1 | Ṁ510 | |
2 | Ṁ101 | |
3 | Ṁ83 |
Sort by:
@derikk after looking at the examples and not getting any correct and then seeing 83% as the human baseline I felt really bad till I read that humans were allowed to Google and use reverse image search.
Related questions
Related questions
Will an AI score over 80% on FrontierMath Benchmark in 2025
29% chance
What will be o3's score on Humanity's Last Exam?
What will be the best normalized score achieved on the original 7 RE-Bench tasks by December 31st 2025?
Will there be a score of 80% or higher on Humanity's Last Exam before April 1, 2025?
3% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
45% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2030?
77% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2027?
76% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2028?
82% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2027?
35% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2030?
60% chance