Will there be an open source LLM as good as GPT4 by June 2024?

Plus

171

Ṁ15k

Jan 1

14%

chance

ALL

I will use my subjective judgement for resolving whether it is as good as GPT-4, but benchmark results will play a part in shaping that judgement. The rest will be qualitative measurement.

Whether something is "open source" is defined liberally here and also will be determined by my subjective judgement, but generally I will deem something open source if (a) anyone can access it and (b) it wasn't the result of an unintentional leak/exfiltration, regardless of the precisions of the license.

I will not personally be trading on this market because it relies on my subjective judgement.

This question is managed and resolved by Manifold.

#AI

#LLMs

Get

1,000

and

3.00

12 Comments

145 Holders

331 Trades

Sort by:

Which GPT-4?

bought Ṁ50 YES

@firstuserhere does it need to be as good as the best GPT-4, or just as good as any of the GPT-4 models?

Would you have called Llama open source and would it have resolved this market to yes if it was as good as you want?

@Seeker Yes, i know LLaMA isn't truly open source but it would've qualified for the purposes of this market.

i find it interesting that this market is at ~27% although mine is at ~70% - mine focuses on the full year and only relies on one metric to resolve while this one is only till june and relies a bit on subjective definition so maybe this is why

The extended version of this market, for the entire 2024

Why does this close in January?

I think an Elo ranking (https://arena.lmsys.org/) could be used to determine the winner objectively. Interestingly, Mistral-Medium is on par with GPT 3.5 in terms of elo :O

'Mistral-Medium outperforms GPT-4 in Winogrande benchmark lmao'

https://twitter.com/yupiop12/status/1734137238177698106

bought Ṁ62 YES

@Dom95cc The Mistral/Mixtral models only seem good in very particular ways. Medium only scores 75 on the MMLU.

@firstuserhere GPT-4 as it is then or as it is at market open?

@TobiasH the benchmark results for GPT-4 from its report at the time of release, and qualitative baseline of today's GPT-4

Related questions

Related questions