HumanEval 90% #1: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2024?
13
250Ṁ1997resolved Oct 22
Resolved
YES1H
6H
1D
1W
1M
ALL
Benchmark link: https://paperswithcode.com/sota/code-generation-on-humaneval
pass@1 means the model gets a single attempt.
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ396 | |
| 2 | Ṁ268 | |
| 3 | Ṁ56 | |
| 4 | Ṁ30 | |
| 5 | Ṁ21 |
Sort by:
People are also trading
Related questions
Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?
67% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
61% chance
What will be the best AI performance on Humanity's Last Exam by December 31st 2025?
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2030?
77% chance
What will be the best performance on EnigmaEval by December 31st 2025?
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
17% chance
Will Al achieve 95% or higher on the Humanity's Last Exam benchmark before 2030?
60% chance
Will an AI achieve >80% performance on the FrontierMath benchmark before 2027?
74% chance
Will an AI score over 80% on FrontierMath Benchmark in 2025
21% chance
Will Al achieve 85% or higher on the Humanity's Last Exam benchmark before 2028?
75% chance