In what year will a GPT3-equivalent model be able to run on consumer hardware?
6
resolved Apr 3
Resolved
2023

"Consumer hardware" is defined as costing no more than $3,000 USD for everything that goes inside the case (not including peripherals).

In terms of "GPT3-equivalent model," I'll go with whatever popular consensus seems to indicate the top benchmarks (up to three) are regarding performance. The performance metrics should be within 10% of GPT3's. In the absence of suitable benchmarks I'll make an educated guess come resolution time after consulting educated experts on the subject.

All that's necessary is for the model to run inference, and it doesn't matter how long it takes to generate output so long as you can type in a prompt and get a reply in less than 24 hours. So in the case GPT3's weights are released and someone is able to shrink that model down to run on consumer hardware and get any output at all in less than a day, and the performance of the output meets benchmarks, this resolves to whatever year that first happens in.

🏅 Top traders

#NameTotal profit
1Ṁ13
2Ṁ6
3Ṁ4
4Ṁ0
Sort by:
LarsDoucet avatar
Lars Doucet

I've seen enough. Not sure if anything that runs locally is GPT-3.*5* equivalent, but it seems clear we have GPT-3 equivalent stuff right now. And llama.cpp has had a lot of advancements just in the last week in terms of running even faster locally.

firstuserhere avatar
firstuserherepredicted LOWER at 2024
firstuserhere avatar
firstuserherepredicted LOWER at 2024

@firstuserhere

1. "In our experiments, we employed the gpt-3.5-turbo and text-davinci-003 variants of the GPT models as the large language models"

2. "we use ChatGPT to conduct task planning when receiving a user request, select models according to their function descriptions available in HuggingFace, execute each subtask with the selected AI model, and summarize the response according to the execution results."

3. "We built HuggingGPT to tackle generalized AI tasks by integrating the HuggingFace hub with 400+ task-specific models around ChatGPT."

LarsDoucet avatar
Lars Doucet

What say you hive-mind, is this GPT-3 equivalent?
https://vicuna.lmsys.org/

firstuserhere avatar
firstuserherepredicted LOWER at 2023

@LarsDoucet can resolve

Gigacasting avatar
Gigacasting

Llama runs on MacBook pro and can beat gpt3 on a phone too

LarsDoucet avatar
Lars Doucet

@Gigacasting Link me to the paper source a repo where I can run it on my machine? (I'm inclined to believe you, just want to verify a bit)

firstuserhere avatar
firstuserherebought Ṁ50 of LOWER

@LarsDoucet stanford alpaca is qualitatively equivalent to TDV3

ian avatar
Ian Philipsbought Ṁ100 of LOWER
LarsDoucet avatar
Lars Doucet

@ian Interesting, have any benchmarks been run?

ian avatar
Ian Philipspredicted LOWER at 2223

@LarsDoucet Not that I can tell, yet. Here's the announcement: https://crfm.stanford.edu/2023/03/13/alpaca.html

firstuserhere avatar
firstuserherebought Ṁ200 of LOWER

@ian i haven't used it yet but LLaMa 65B has been run on MacBook pro also

BTE avatar
Brian T. Edwards

@LarsDoucet I am fairly confident this is doable right now. The model runs on CPUs after it is trained after all.

Related markets

In what year will a GPT4-equivalent model be able to run on consumer hardware?2025
In what year will a GPT4-equivalent model be able to run on consumer hardware?2024
Will a GPT4-equivalent model be able to run on consumer hardware before 2024?28%
Will a large GPT-4 equivalent competitor model be revealed by the end of 2023?52%
Will a GPT-3 quality model be trained for under $1,000 by 2030?74%
Will a GPT-3 quality model be trained for under $10.000 by 2030?81%
Will we have an open-source model that is equivalent GPT-4 by end of 2025?82%
Will we have a FOSS GPT-4 Equivalent by the end of the year?15%
Will a GPT-4 quality model be trained for under $10.000 by 2030?78%
Will a model be trained using at least as much compute as GPT-3 using AMD GPUs before Jan 1 2026?75%
Will we have a FOSS equivalent of GPT-4 by the end of the year?12%
Will a version of GPT-4 finetuned for code be released this year?78%
Will mechanistic interpretability be essentially solved for GPT-3 before 2030?15%
Will GPT-4's parameter count be known by end of 2024?45%
Will I be able to use base GPT-4 at any time in 2023?28%
Will there be a fully self-sustaining Auto-GPT agent in 2023?25%
When will GPT5 be announced?
Will GPT-4 be public during 2023?88%
Will a single model running on a single consumer GPU (<1.5k 2020 USD) outperform GPT-3 175B on all benchmarks in the original paper by 2025?91%
When will GPT-5 be released? (2025)30%