How many parameters will GPT-4 have?

2025

GPT-3 has a staggering 175 BILLION parameters

To put it into context
Hugging face's 176 Billion parameter model took 3.5 months on 384 top-of-the-line GPUs to train...

GPT-3 is also over 2 years old.

Nov 17, 1:56am: ~~How many parameters with GPT-4 have?~~ → How many parameters will GPT-4 have?

Science

GPT-4 speculation

>1600

32%

801-1600

19%

301-349 or <300

18%

801-1000

1001-1200

1401-1600

1201-1400

501-550

350-400

1.9%

401-450

1.5%

701-800

1.4%

551-600

1.4%

Add your answer

Sort by:

Noa Nabeshima

I also notice that there are answers that overlap with each other @JustinKwong

Noa Nabeshima

@NoaNabeshima oops, @JustinTorre

Noa Nabeshima

The units for these answers are in B of parameters, right??? @JustinTorre

firstuserhere

@NoaNabeshima lmao

Noa Nabeshimabought Ṁ10 of 301-349 or <300

If GPT-4 is a MoE model it will probably have >1600B parameters.

Neel Nanda

Hugging face's 176 Billion parameter model took 3.5 months on 384 top-of-the-line GPUs to train...

Note that HuggingFace's model was trained on 350B tokens. The Chinchilla optimal amount of tokens for a 175B model is 3500B tokens, so 10x as much!

Andrew Conner

Started one with a numeric answer:

Max Ghenis

Nikola

in billions, I assume?

Nikola

@Nikola Bars (300-350, 350-400, 400-450) seem better for this kind of market, right?

Nikolabought Ṁ1 of 350-400

@Nikola Or just a market for a numerical value

Lavander bought Ṁ3 801-1600 at 19%

Lavander bought Ṁ7 801-1600 at 19%

CuriousNoob bought Ṁ5 801-1600 from 18% to 19%

Floris van Doorn bought Ṁ10 301-349 or <300 from 17% to 18%

firstuserhere sold Ṁ31 301-349 or <300 from 19% to 17%

firstuserhere sold Ṁ2 501-550 at 2%

firstuserhere sold Ṁ6 301-349 or <300 at 19%

firstuserhere sold Ṁ1 451-500 from 0.6% to 0.5%

firstuserhere sold Ṁ1 401-450 from 1.6% to 1.4%

firstuserhere sold Ṁ2 350-400 from 2% to 1.9%

NexVeridian sold Ṁ58 301-349 or <300 from 22% to 19%

Pranav bought Ṁ100 301-349 or <300 from 17% to 22%

Lavander sold Ṁ10 801-1600 from 19% to 18%

Lavander bought Ṁ10 801-1600 from 18% to 19%

Jacob Pfau sold Ṁ5 801-1600 from 19% to 18%

Jacob Pfau sold Ṁ14 >1600 from 33% to 32%

NexVeridian bought Ṁ50 301-349 or <300 from 14% to 17%

rockenots sold Ṁ24 801-1600 from 20% to 19%

rockenots sold Ṁ38 >1600 from 35% to 33%

nmehndir sold Ṁ20 601-700 from 2% to 0.7%

nmehndir sold Ṁ20 701-800 from 3% to 1.4%

nmehndir sold Ṁ20 551-600 from 3% to 1.3%

nmehndir bought Ṁ20 551-600 from 1.3% to 3%

nmehndir bought Ṁ20 701-800 from 1.4% to 3%

nmehndir bought Ṁ20 601-700 from 0.7% to 2%

nmehndir bought Ṁ50 1001-1200 from 3% to 6%

nmehndir bought Ṁ50 801-1000 from 6% to 10%

nmehndir bought Ṁ10 701-800 from 0.8% to 1.5%

nmehndir bought Ṁ10 551-600 from 0.8% to 1.5%

nmehndir bought Ṁ50 1401-1600 from 0.0% to 3%

nmehndir bought Ṁ50 1201-1400 from 0.0% to 3%

nmehndir bought Ṁ50 1001-1200 from 0.0% to 4%

nmehndir bought Ṁ10 601-700 from 0.2% to 0.9%

nmehndir bought Ṁ10 551-600 from 0.2% to 0.9%

nmehndir bought Ṁ10 701-800 from 0.2% to 0.9%

Noa Nabeshima bought Ṁ100 >1600 from 39% to 43%

nmehndir bought Ṁ100 801-1000 from 0.6% to 8%

Noa Nabeshima bought Ṁ100 >1600 from 37% to 42%

Noa Nabeshima bought Ṁ100 >1600 from 31% to 37%

nmehndir bought Ṁ25 501-550 from 1.6% to 4%

nmehndir bought Ṁ10 501-550 from 0.8% to 1.6%

Noa Nabeshima sold Ṁ5 350-400 at 4%

Noa Nabeshima bought Ṁ100 >1600 from 26% to 32%

Lavander bought Ṁ4 801-1600 at 37%

1a3orn sold Ṁ1 350-400 at 5%

1a3orn sold Ṁ3 301-349 or <300 at 25%

1a3orn sold Ṁ3 350-400 at 5%

1a3orn sold Ṁ7 301-349 or <300 from 26% to 25%

1a3orn sold Ṁ4 501-550 from 1.2% to 0.9%

1a3orn sold Ṁ3 451-500 from 1.5% to 1.1%

Related markets

Andrew Conner

How many parameters will GPT-4 have?650B

Embedded Agent

Will GPT-4 have over 1 trillion parameters?58%

Max Ghenis

Will GPT-4 have at least 100 trillion parameters?1%

Mira

Will GPT-5 have over 1 trillion parameters?83%

nmehndir

Will GPT-4's parameter count be publicly announced by the end of March?3%

Jack

GPT-4 have 500b+ parameters?87%

ada

Will GPT-4's parameter count be announced by the end of 2023?26%

Vincent Luczkow

GPT-4 #5: Will GPT-4 be a dense model?30%

Related markets

Andrew Conner

How many parameters will GPT-4 have?650B

Embedded Agent

Will GPT-4 have over 1 trillion parameters?58%

Max Ghenis

Will GPT-4 have at least 100 trillion parameters?1%

Mira

Will GPT-5 have over 1 trillion parameters?83%

nmehndir

Will GPT-4's parameter count be publicly announced by the end of March?3%

Jack

GPT-4 have 500b+ parameters?87%

ada

Will GPT-4's parameter count be announced by the end of 2023?26%

Vincent Luczkow

GPT-4 #5: Will GPT-4 be a dense model?30%