Will there be an open source LLM as good as GPT4 by June 2024?
172
1.5kṀ15k
Jan 1
12%
chance

I will use my subjective judgement for resolving whether it is as good as GPT-4, but benchmark results will play a part in shaping that judgement. The rest will be qualitative measurement.

Whether something is "open source" is defined liberally here and also will be determined by my subjective judgement, but generally I will deem something open source if (a) anyone can access it and (b) it wasn't the result of an unintentional leak/exfiltration, regardless of the precisions of the license.

I will not personally be trading on this market because it relies on my subjective judgement.

AI
LLMs
Get
Ṁ1,000
to start trading!

People are also trading

Will an open-source fully functional Auto-GPT like LLM exist by the end of 2025?
89% chance
Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?
73% chance
Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?
82% chance
Which next-gen frontier LLMs will be released before GPT-5? (2025)
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
13% chance
China will make a LLM approximately as good or better than GPT4 before 2025
89% chance
an LLM as capable as GPT-4 runs on one 4090 by March 2025
35% chance
How much time will pass between an LLM being released that beats GPT4 and the next OpenAI LLM being released? (+ANSWERS)
Will it be possible to run an LLM of GPT-4 (or higher) capability on a portable device by 2027?
47% chance
an LLM as capable as GPT-4 runs on one 3090 by March 2025
33% chance
Sort by:

Which GPT-4?

bought Ṁ50 YES

@firstuserhere does it need to be as good as the best GPT-4, or just as good as any of the GPT-4 models?

Would you have called Llama open source and would it have resolved this market to yes if it was as good as you want?

@Seeker Yes, i know LLaMA isn't truly open source but it would've qualified for the purposes of this market.

i find it interesting that this market is at ~27% although mine is at ~70% - mine focuses on the full year and only relies on one metric to resolve while this one is only till june and relies a bit on subjective definition so maybe this is why

The extended version of this market, for the entire 2024

Why does this close in January?

I think an Elo ranking (https://arena.lmsys.org/) could be used to determine the winner objectively. Interestingly, Mistral-Medium is on par with GPT 3.5 in terms of elo :O

'Mistral-Medium outperforms GPT-4 in Winogrande benchmark lmao'

https://twitter.com/yupiop12/status/1734137238177698106

@Dom95cc The Mistral/Mixtral models only seem good in very particular ways. Medium only scores 75 on the MMLU.

@firstuserhere GPT-4 as it is then or as it is at market open?

@TobiasH the benchmark results for GPT-4 from its report at the time of release, and qualitative baseline of today's GPT-4

People are also trading

Will an open-source fully functional Auto-GPT like LLM exist by the end of 2025?
89% chance
Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?
73% chance
Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?
82% chance
Which next-gen frontier LLMs will be released before GPT-5? (2025)
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
13% chance
China will make a LLM approximately as good or better than GPT4 before 2025
89% chance
an LLM as capable as GPT-4 runs on one 4090 by March 2025
35% chance
How much time will pass between an LLM being released that beats GPT4 and the next OpenAI LLM being released? (+ANSWERS)
Will it be possible to run an LLM of GPT-4 (or higher) capability on a portable device by 2027?
47% chance
an LLM as capable as GPT-4 runs on one 3090 by March 2025
33% chance
© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules