Will Meta release an open source language model that outperforms GPT-4 by the end of 2024
22
1kṀ1035
resolved Jan 3
Resolved
YES

Will resolve to YES if Meta releases an open source model that acheives a higher average score than GPT-4 on the following benchmarks by the end of 2024:

HellaSwag (few-shot): 0.953

MMLU (few-shot): 0.864

AI2 Reasoning Challenge (ARC): 0.963

  • Update 2025-03-01 (PST) (AI summary of creator comment): - Llama 3.1 405B achieves the following benchmark scores:

    • MMLU (zero-shot CoT): 0.886

    • ARC (zero-shot): 0.969

    • Hellaswag score is not reported due to potential contamination and is not considered in the resolution criteria.

    • The model is deemed open-source, and based on its performance and higher Elo on LMSYS, the market is resolved to YES.

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ136
2Ṁ78
3Ṁ66
4Ṁ45
5Ṁ30
© Manifold Markets, Inc.TermsPrivacy