How many parameters will GPT-4 have?

GPT-3 has a staggering 175 BILLION parameters


To put it into context
Hugging face's 176 Billion parameter model took 3.5 months on 384 top-of-the-line GPUs to train...

GPT-3 is also over 2 years old.

Nov 17, 1:56am: How many parameters with GPT-4 have? → How many parameters will GPT-4 have?

Nikola avatar>1600
32%
Phi avatar801-1600
19%
NoaNabeshima avatar301-349 or <300
18%
Nikola avatar801-1000
9%
Nikola avatar1001-1200
6%
Nikola avatar1401-1600
3%
Nikola avatar1201-1400
3%
Nikola avatar501-550
2%
Nikola avatar350-400
1.9%
Nikola avatar401-450
1.5%
Nikola avatar701-800
1.4%
Nikola avatar551-600
1.4%
Add your answer
Sort by:
NoaNabeshima avatar
Noa Nabeshima

I also notice that there are answers that overlap with each other @JustinKwong

NoaNabeshima avatar
Noa Nabeshima

The units for these answers are in B of parameters, right??? @JustinTorre

firstuserhere avatar
firstuserhere
NoaNabeshima avatar
Noa Nabeshimabought Ṁ10 of 301-349 or <300

If GPT-4 is a MoE model it will probably have >1600B parameters.

NeelNanda avatar
Neel Nanda

Hugging face's 176 Billion parameter model took 3.5 months on 384 top-of-the-line GPUs to train...

Note that HuggingFace's model was trained on 350B tokens. The Chinchilla optimal amount of tokens for a 175B model is 3500B tokens, so 10x as much!

andrew avatar
Andrew Conner

Started one with a numeric answer:

MaxGhenis avatar
Max Ghenis
Nikola avatar
Nikola

in billions, I assume?

Nikola avatar
Nikola

@Nikola Bars (300-350, 350-400, 400-450) seem better for this kind of market, right?

Nikola avatar
Nikolabought Ṁ1 of 350-400

@Nikola Or just a market for a numerical value