HumanEval 90% #2: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2025?
8
Ṁ190Ṁ443resolved Jun 10
Resolved
YES1H
6H
1D
1W
1M
ALL
Benchmark link: https://paperswithcode.com/sota/code-generation-on-humaneval
pass@1 means the model gets a single attempt.
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Trader | Total profit |
|---|---|---|
| 1 | Ṁ20 | |
| 2 | Ṁ8 | |
| 3 | Ṁ2 | |
| 4 | Ṁ0 |
People are also trading
Related questions
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2027?
32% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2027?
7% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2030?
87% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2030?
36% chance
What will be the best performance on EnigmaEval by December 31st 2026?
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2028?
36% chance
In what year will AI achieve a score of 95% or higher on the PhysBench leaderboard?
2036
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2028?
79% chance
In what year will Al achieve 95% or higher score on the Humanity’s Last Exam benchmark?
2029
In what year will Al achieve 95% or higher score on the Humanity’s Last Exam benchmark?