HumanEval 90% #1: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2024?
13
250Ṁ1997
resolved Oct 22
Resolved
YES

Benchmark link: https://paperswithcode.com/sota/code-generation-on-humaneval

pass@1 means the model gets a single attempt.

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ396
2Ṁ268
3Ṁ56
4Ṁ30
5Ṁ21
Sort by:

Unless I've misunderstood what this market is asking, it seems like the answer is yes. At the linked benchmark page, the top model is currently sitting at 94.4%

@js Thank you for pointing this out

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules