In yottaFLOPs (10^24), how much compute will GPT-4 be trained with?

1kṀ791

2027

expected

ALL

Resolves N/A if the number is not public/calculateable from public information, or if GPT-4 is not released by market close.

Technical AI Timelines

GPT-4 speculation

Get

1,000

to start trading!

People are also trading

How much compute will be used to train GPT-5?

GPT-5 trained with >=24k GPUs?

82% chance

Will GPT-4 be trained on more than 10T text tokens?

36% chance

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

30% chance

If we find out in 2024, was o1's Transformer base trained on 10+x as much compute as GPT-4's?

19% chance

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

82% chance

What hardware will GPT-5 be trained on?

Was GPT-4 trained in 4 months or less?

59% chance

GPT-4 performance and compute efficiency from a simple architecture before 2026

19% chance

How many parameters does GPT4o have?

7 Comments

14 Holders

41 Trades

Sort by:

It seems very unlikely that the release of GPT-4 will resolve this. However the market does not close until 2027, and I will leave it open until then in case the information is either (credibly) leaked or OpenAI decides to release it after the fact. If neither of those occurs the market will resolve N/A

opened a Ṁ1,000 HIGHER at 22% order

Hey, I think this market should be resolved. From what I can tell OpenAI people treated the semianalysis leak (https://www.semianalysis.com/p/gpt-4-architecture-infrastructure) seriously and it reports 280B active parameters for 13 trillion tokens, so 13T tokens * 0.28T params x 6 flops/token/param = 21.84 x 10^24 flops.

I think the semianalysis leak is probably accurate, but I'm being cautious about resolving from leaks. I don't think there's any rush to resolve this now when I can just wait and see if more information comes out?

hmm, I find this leak highly credible because ~every AI person I know takes it seriously. Seems fine to wait though.

opened a Ṁ500 LOWER at 23% order

@vluzko I think it's pretty unlikely at this point that new information will come out

(Epistemic status: I’m doing all this calculation from memory so might be wrong) GPT-3 was trained on 3.2e23 FLOPs (general calculation is C=6ND, N number of params and D datasize), I think we should expect around twice the size and 50x the dataset size, in which case 3.2e25 FLOPs should be reasonable for GPT-3. This would imply a resolution of 32 to this question. 150 is way too high imo