HumanEval 90% #2: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2025?
8
190Ṁ443resolved Jun 10
Resolved
YES1H
6H
1D
1W
1M
ALL
Benchmark link: https://paperswithcode.com/sota/code-generation-on-humaneval
pass@1 means the model gets a single attempt.
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ20 | |
| 2 | Ṁ8 | |
| 3 | Ṁ2 | |
| 4 | Ṁ0 |
People are also trading
Related questions
What will be the best performance on EnigmaEval by December 31st 2025?
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
61% chance
What will be the best AI performance on Humanity's Last Exam by December 31st 2025?
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2030?
77% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2030?
60% chance
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
17% chance
Will an AI score over 80% on FrontierMath Benchmark in 2025
21% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2028?
75% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2027?
39% chance
Will an AI achieve >80% performance on the FrontierMath benchmark before 2027?
74% chance