![](/_next/image?url=https%3A%2F%2Ffirebasestorage.googleapis.com%2Fv0%2Fb%2Fmantic-markets.appspot.com%2Fo%2Fdream%252FDKQqW4X4lp.png%3Falt%3Dmedia%26token%3D633b88a2-362a-4961-92f4-898c2e23f0e8&w=3840&q=75)
HumanEval 90% #3: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2026?
Mini
8
Ṁ367resolved Jun 10
Resolved
YES1D
1W
1M
ALL
Benchmark link: https://paperswithcode.com/sota/code-generation-on-humaneval
pass@1 means the model gets a single attempt.
Get Ṁ600 play money
🏅 Top traders
# | Name | Total profit |
---|---|---|
1 | Ṁ35 | |
2 | Ṁ21 | |
3 | Ṁ11 | |
4 | Ṁ4 | |
5 | Ṁ2 |
Related questions
Related questions
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
AI resolves at least X% on SWE-bench without any assistance, by 2028?
AI resolves at least X% on SWE-bench assistance, by 2025?
Will >50% of the tasks in the WebArena benchmark be solved by EOY 2024?
62% chance
Will an AI be capable of achieving a perfect score on the Putnam exam before 2030?
41% chance
Will AI pass Video Turing Test by 2030?
68% chance
Human-machine intelligence parity achieved before 2028
51% chance
Human-machine intelligence parity achieved before 2030
70% chance
By 2025, will most well-educated people expect AI to within 10 years be better at intellectual work than 99% of humans?
15% chance
Will effective personality simulation be available by 2030?
54% chance