Will a 15 billion parameter LLM match or outperform GPT4 in 2024?
Standard
15
Ṁ835
Dec 31
24%
chance

GPT-4's benchmark results as of its release in march 2023.

Acceptable upto 17 billion.

Get
Ṁ1,000
and
S1.00
Sort by:

@mods can this resolve yes? According to LMSYS, the March 2024 version of GPT-4 is currently ranked 40th and is outclassed by Gemini 1.5 8b in 31st and Gemma 2 9b in 25.

@JonathanMilligan Closing while I try and understand.

@JonathanMilligan Wait, why should LMSYS count? That isn't a benchmark, right?

@NathanpmYoung Yeah LMSYS is a benchmark of how people rank the output of LLM models.

@JonathanMilligan Currently it seems to be the best benchmark of real “in the wild” use.

@JonathanMilligan Okay but it isn't a "benchmark" see "GPT-4's benchmark results as of its release in march 2023." Right?

@mods what do you think?

Sebastian Bubeck (Microsoft research) recently said that he thinks a 13 billion parameter model could do this. There've been reports of others working at similar sized models for similar performance.

@firstuserhere Source? Interesting speculation there