xAI Grok will beat OpenAI's flagship model on HumanEval benchmarks by the end of 2024.
101
Ṁ1.1kṀ14kresolved Jun 9
Resolved
NO1H
6H
1D
1W
1M
ALL
This is inclusive of any new models OpenAI unveils in 2024, but the question resolves to "yes" if Grok beats OpenAI at any time in 2024 against their current state of the art model.
This question is managed and resolved by Manifold.
Market context
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Trader | Total profit |
|---|---|---|
| 1 | Ṁ538 | |
| 2 | Ṁ124 | |
| 3 | Ṁ72 | |
| 4 | Ṁ50 | |
| 5 | Ṁ45 |
People are also trading
Sort by:
@DanMan314 67.0% is the HumanEval figure from the original GPT-4 report published more than a year ago. The current zero-shot GPT-4 performance, as reported by Papers With Code, is 76.5%, which is from Guo et al. (January 2024).
Note that the market creator is banned, so this will probably be resolved by moderators. Personally, I think the current version of GPT-4 is the more natural interpretation of "OpenAI's flagship model" than the original version of GPT-4.