Will Meta release an open source language model that outperforms GPT-4 by the end of 2024
22
1kṀ1035resolved Jan 3
Resolved
YES1H
6H
1D
1W
1M
ALL
Will resolve to YES if Meta releases an open source model that acheives a higher average score than GPT-4 on the following benchmarks by the end of 2024:
HellaSwag (few-shot): 0.953
MMLU (few-shot): 0.864
AI2 Reasoning Challenge (ARC): 0.963
Update 2025-03-01 (PST) (AI summary of creator comment): - Llama 3.1 405B achieves the following benchmark scores:
MMLU (zero-shot CoT): 0.886
ARC (zero-shot): 0.969
Hellaswag score is not reported due to potential contamination and is not considered in the resolution criteria.
The model is deemed open-source, and based on its performance and higher Elo on LMSYS, the market is resolved to YES.
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
🏅 Top traders
# | Name | Total profit |
---|---|---|
1 | Ṁ136 | |
2 | Ṁ78 | |
3 | Ṁ66 | |
4 | Ṁ45 | |
5 | Ṁ30 |
People are also trading
Related questions
Will we have an open-source model that is equivalent GPT-4 by end of 2025?
82% chance
Will there be an AI language model that strongly surpasses ChatGPT and other OpenAI models before the end of 2025?
45% chance
Will OpenAI announce a model with a name containing the string "GPT-4b" in 2025
16% chance
Will there be a language model called GPT-5, released by OpenAI, this decade?
95% chance
Will a language model that runs locally on a consumer cellphone beat GPT4 by EOY 2026?
84% chance