Will we have an open-source model that is equivalent GPT-4 by end of 2025?
Will we have an open-source model that is equivalent GPT-4 by end of 2025?
23
1kṀ978
2026
82%
chance

Similar to /LoganZoellner/will-we-have-a-foss-equivalent-of-g

With this much excitement and energy in the LLM space, I am fairly confident in this prediction: Before the end of this year, there will be an open source model that is equivalent in quality to #GPT4

Resolves positive if a FOSS model (no signup required to download) is pubically available by 12/31/25 that outperforms GPT-4 on at least 50% of the tests listed in this article (https://www.semafor.com/article/03/15/2023/how-gpt-4-performed-in-academic-exams)

Get
Ṁ1,000
to start trading!


Sort by:
1y

Could make sense to base the resolution criteria off of ML benchmarks instead of academic/professional exams, since most open source models have not been evaluating on these exams, and there are not very many details on the specific exams/questions used. Some common benchmarks are HellaSwag, MMLU, and the AI2 Reasoning Challenge, which are listed in the GPT4 technical report, and I am using in my market: https://manifold.markets/tk/will-meta-release-an-open-source-la

Will Meta release an open source language model that outperforms GPT-4 by the end of 2024
33% chance. Will resolve to YES if Meta releases an open source model that acheives a higher average score than GPT-4 on the following benchmarks by the end of 2024: HellaSwag (few-shot): 0.953 MMLU (few-shot): 0.864 AI2 Reasoning Challenge (ARC): 0.963
1y

The question and description contradict each other. End of 2023, or end of 2025?

1y

@BionicD0LPH1N fixed 2025

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.

Related questions

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules