Will a single model running on a single consumer GPU (<1.5k 2020 USD) outperform GPT-3 175B on all benchmarks in the original paper by 2025?
29
1kṀ3926
2026
86%
chance
There are no restrictions on the amount or kind of compute used to *train* the model. Question is about whether it will actually be done, not whether it will be possible in theory. If I judge the model to really be many specific models stuck together to look like one general model it will not count.
AI
Technical AI Timelines
Get
Ṁ1,000
to start trading!

People are also trading

Size of smallest open-source LLM marching GPT 3.5's performance in 2025? (GB)
2.16
Will a model be trained using at least as much compute as GPT-3 using AMD GPUs before Jan 1 2026?
84% chance
Will a model as great as GPT-5 be available to the public in 2025?
84% chance
Will a language model that runs locally on a consumer cellphone beat GPT4 by EOY 2026?
78% chance
In what year will a GPT4-equivalent model be able to run on consumer hardware?
2026
In what year will a GPT4-equivalent model be able to run on consumer hardware?
2026
Will a GPT-3 quality model be trained for under $1,000 by 2030?
82% chance
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
13% chance
Will a GPT-3 quality model be trained for under $10.000 by 2030?
83% chance
GPT-5 trained with >=24k GPUs?
82% chance
Sort by:

Llamas on pixel 7s https://github.com/rupeshs/alpaca.cpp/tree/linux-android-build-support (ik ik its not over 13B yet, just sharing progress)

predictedYES

There are people who run 30B Llama on consumer PC successfully and even 65B (but it is extremely slow)

@ValeryCherepanov By "run on a single GPU" I mean the weights + one full input vector can fit on a consumer GPU at once. Otherwise the question would be meaningless - you can always split up matrices into smaller blocks and run the computation sequentially.

This is now extremely close to being resolved by Llama (Llama 13B does not actually beat GPT-3 on every measured benchmark, however, it only comes very close). 72% is way too low though so I guess whoever reads this comment first can collect some free mana in expectation.

@vluzko

Are the benchmarks text generation only? Or do they work with chat models too

FLAN-T5 3B very likely can resolve this now, but I suspect it will be a while before anyone actually bothers to run it on all of the benchmarks.

lol yeah this one's gonna happen: https://arxiv.org/abs/2205.05131

Tautologically this is possible today. Whether it’s possible to do “with entire model in GPU memory at once” just not that interesting to calculate.
I think it's a tight call. I think I'd go the other way if this was for 2027.

People are also trading

Size of smallest open-source LLM marching GPT 3.5's performance in 2025? (GB)
2.16
Will a model be trained using at least as much compute as GPT-3 using AMD GPUs before Jan 1 2026?
84% chance
Will a model as great as GPT-5 be available to the public in 2025?
84% chance
Will a language model that runs locally on a consumer cellphone beat GPT4 by EOY 2026?
78% chance
In what year will a GPT4-equivalent model be able to run on consumer hardware?
2026
In what year will a GPT4-equivalent model be able to run on consumer hardware?
2026
Will a GPT-3 quality model be trained for under $1,000 by 2030?
82% chance
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
13% chance
Will a GPT-3 quality model be trained for under $10.000 by 2030?
83% chance
GPT-5 trained with >=24k GPUs?
82% chance
© Manifold Markets, Inc.•Terms•Privacy