
GPT-3 has a staggering 175 BILLION parameters
To put it into context
Hugging face's 176 Billion parameter model took 3.5 months on 384 top-of-the-line GPUs to train...
GPT-3 is also over 2 years old.
Nov 17, 1:56am: How many parameters with GPT-4 have? → How many parameters will GPT-4 have?
32%
19%
18%
9%
6%
3%
3%
2%
1.9%
1.5%
1.4%
1.4%
Add your answer
Sort by:
Noa Nabeshimabought Ṁ10 of 301-349 or <300
If GPT-4 is a MoE model it will probably have >1600B parameters.
Neel Nanda
Hugging face's 176 Billion parameter model took 3.5 months on 384 top-of-the-line GPUs to train...
Note that HuggingFace's model was trained on 350B tokens. The Chinchilla optimal amount of tokens for a 175B model is 3500B tokens, so 10x as much!


































Related markets
Andrew Conner
How many parameters will GPT-4 have?650B
Embedded Agent
Will GPT-4 have over 1 trillion parameters?58%
Max Ghenis
Will GPT-4 have at least 100 trillion parameters?1%
Mira
Will GPT-5 have over 1 trillion parameters?83%

nmehndir
Will GPT-4's parameter count be publicly announced by the end of March?3%

Jack
GPT-4 have 500b+ parameters?87%
ada
Will GPT-4's parameter count be announced by the end of 2023?26%

Vincent Luczkow
GPT-4 #5: Will GPT-4 be a dense model?30%
Related markets
Andrew Conner
How many parameters will GPT-4 have?650B
Embedded Agent
Will GPT-4 have over 1 trillion parameters?58%
Max Ghenis
Will GPT-4 have at least 100 trillion parameters?1%
Mira
Will GPT-5 have over 1 trillion parameters?83%

nmehndir
Will GPT-4's parameter count be publicly announced by the end of March?3%

Jack
GPT-4 have 500b+ parameters?87%
ada
Will GPT-4's parameter count be announced by the end of 2023?26%

Vincent Luczkow
GPT-4 #5: Will GPT-4 be a dense model?30%