Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time? | Manifold

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

40

1kṀ6753

Jun 2

30%

chance

1H

6H

1D

1W

1M

ALL

This question resolves YES if GPT-4 has enough data to roughly match the best-known scaling laws prescriptions known at the time of the training of GPT-4. Currently, this would mean following Chinchilla scaling laws. By roughly, I mean that it can be off by 20%. That is, if GPT-4 is 100B parameters, which would prescribe 12T tokens as per (currently known) optimal scaling laws, GPT-4 would need to be trained from ~10T to ~14T tokens for this question to resolve positively.

GPT-4 speculation

New Year's Resolutions 2024

Get

1,000

to start trading!

People are also trading

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

In yottaFLOPs (10^24), how much compute will GPT-4 be trained with?

Will a GPT-4 level system be trained for <$1mm by 2028?

GPT-4 performance and compute efficiency from a simple architecture before 2026

Will a GPT-4 level system be trained for <$1mm by 2030?

Will a GPT-4 quality model be trained for under $10.000 by 2030?

GPT-4 #5: Will GPT-4 be a dense model?

Will growth in the maximum MW used to train AIs, slow down by more than x2 after GPT-5-like?

In what year will a GPT4-equivalent model be able to run on consumer hardware?

In what year will a GPT4-equivalent model be able to run on consumer hardware?

Related questions

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

In yottaFLOPs (10^24), how much compute will GPT-4 be trained with?

Will a GPT-4 level system be trained for <$1mm by 2028?

GPT-4 performance and compute efficiency from a simple architecture before 2026

Will a GPT-4 level system be trained for <$1mm by 2030?

Will a GPT-4 quality model be trained for under $10.000 by 2030?

GPT-4 #5: Will GPT-4 be a dense model?

Will growth in the maximum MW used to train AIs, slow down by more than x2 after GPT-5-like?

In what year will a GPT4-equivalent model be able to run on consumer hardware?

In what year will a GPT4-equivalent model be able to run on consumer hardware?

© Manifold Markets, Inc.•Terms•Privacy