xAI Grok will beat OpenAI's flagship model on HumanEval benchmarks by the end of 2024.
Plus
96
Ṁ12kJan 1
15%
chance
1D
1W
1M
ALL
This is inclusive of any new models OpenAI unveils in 2024, but the question resolves to "yes" if Grok beats OpenAI at any time in 2024 against their current state of the art model.
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Sort by:
bought Ṁ10 YES from 69% to 71%
@DanMan314 67.0% is the HumanEval figure from the original GPT-4 report published more than a year ago. The current zero-shot GPT-4 performance, as reported by Papers With Code, is 76.5%, which is from Guo et al. (January 2024).
Note that the market creator is banned, so this will probably be resolved by moderators. Personally, I think the current version of GPT-4 is the more natural interpretation of "OpenAI's flagship model" than the original version of GPT-4.
Related questions
Related questions
By the end of Q1 2025 will an open source model beat OpenAI’s o1 model?
76% chance
Will there be an AI language model that strongly surpasses ChatGPT and other OpenAI models before the end of 2024?
9% chance
Will Grok 2 'exceed current [March 28 2024] AI on all metrics'?
15% chance
Will xAI release Grok 3 by the end of 2024?
46% chance
By the end of Q2 2025 will an open source model beat OpenAI’s o1 model?
76% chance
Will Grok 3 be 'the most powerful AI in the world'?
13% chance
Will Grok2 “Exceed Current AI Models on All Metrics” [Creator's best judgement]
19% chance
Will OpenAI be in the lead in the AGI race end of 2026?
44% chance
What exactly will xAI open source, when they "open source" their AI language model "Grok" ?
By the end of Q3 2025 will an open source model beat OpenAI’s o1 model?
77% chance