HumanEval 90% #4: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2027?
3
110Ṁ128resolved Jun 10
Resolved
YES1H
6H
1D
1W
1M
ALL
Benchmark link: https://paperswithcode.com/sota/code-generation-on-humaneval
pass@1 means the model gets a single attempt.
This question is managed and resolved by Manifold.
Market context
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ11 | |
| 2 | Ṁ11 | |
| 3 | Ṁ8 |
People are also trading
Related questions
Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?
64% chance
Will an AI achieve >85% performance on the FrontierMath benchmark before 2027?
37% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2030?
83% chance
Top score on Humanity's Last Exam > 50% by 2027?
95% chance
Top score on Humanity's Last Exam > 50% by 2028?
95% chance
Top score on Humanity's Last Exam > 50% by 2029?
99% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2030?
33% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2027?
7% chance
Will an AI achieve >80% performance on the FrontierMath benchmark before 2027?
50% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2027?
61% chance