By 2028, will there be a language model of less than 10B parameters that is superior to GPT-4?

1kṀ553

2028

84%

chance

ALL

Pretty impressed by this:

https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/

I will not bet in this market. I will look primarily at whatever benchmarks are most prominent at the time to determine which model is better.

The model does not have to be clearly superior to gpt-4, if it's just a bit better I will resolve YES. If anyone has suggestions for less subjective resolution criteria that won't be goodharted I am open.

Technical AI Timelines

GPT-4 speculation

Get

1,000

to start trading!

People are also trading

Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?

13% chance

Will a language model comparable to GPT-4 be trained, with ~1/10th the amount of energy it took train GPT-4, by 2028?

92% chance

By January 2026, will we have a language model with similar performance to GPT-3.5 (i.e. ChatGPT as of Feb-23) that is small enough to run locally on the highest end iPhone available at the time?

93% chance

Will there be an AI language model that strongly surpasses ChatGPT and other OpenAI models before the end of 2025?

46% chance

Will any 10 trillion+ parameter language model that follows instructions be released to the public before 2026?

43% chance

Will we have an open-source model that is equivalent GPT-4 by end of 2025?

82% chance

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

82% chance

Will there be a language model called GPT-5, released by OpenAI, this decade?

95% chance

Will OpenAI announce a model with a name containing the string "GPT-4b" in 2025

64% chance

Will it be possible to disentangle most of the features learned by a model comparable to GPT-4 this decade?

Sort by:

Gemma 2 9b-it is higher than original ChatGPT on llmsys arena. I don’t think it’s better or even on the same level but I think it’s a signal that it’s already close. 27b version I think in the same level

if the model can be stored on 10b params but takes 100000x more time at inference time, because it searches through lots of possibilities or reprompts itself a bunch of times or similar tricks to improve its output, would that still count? In 2023 it would still not be practical to run, but it would still be smaller in the ‘# of parameters‘ sense