Will there be an LLM (as good as GPT-4) that was trained with 1/10th the energy consumed to train GPT-4, by 2026?
Will there be an LLM (as good as GPT-4) that was trained with 1/10th the energy consumed to train GPT-4, by 2026?
resolved by
23
1.4kṀ12k
resolved Jan 10
Resolved
YES

The total power consumption could be estimated to be around 50-60 million kWh for training GPT-4.

1/10th of this energy = 5-6 million kWh

1/100th of this energy = 0.5-0.6 million kWh

See calculations below:

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ521
2Ṁ113
3Ṁ67
4Ṁ27
5Ṁ17


Sort by:

@mods Resolves as YES. The creator's account is deleted, but DeepSeek v3 is much better than the original GPT-4 and was trained with an energy consumption of less than 5 million kWh.

Detailed calculation:
From the paper https://github.com/deepseek-ai/DeepSeek-V3/blob/main/DeepSeek_V3.pdf, we find that DeepSeek V3 required only 2.788 million H800 GPU hours for its full training.

The H800 GPU has a maximum power draw of 0.35 kW, see https://www.techpowerup.com/gpu-specs/h800-pcie-80-gb.c4181

Thus, the GPUs used at most 0.9758 million kWh (= 0.35 kW × 2.788 million hours) during training. Accounting for system power draw and other inefficiencies, we apply a factor of 2 to estimate an upper bound of at most 2 million kWh in total energy consumption for training the model. This is clearly below the 5 million kWh threshold required for resolving this market as YES.

DeepSeek is not only as good as the original GPT-4 but is much better, see https://lmarena.ai/




@mods It has been a few days, and there are no objections to the resolution proposal above. The calculation is not rocket science. Due to sanctions, the Chinese face significant compute limitations, so they designed an extremely efficient model that can be trained using less than 4% of the compute required for the original GPT-4. This model easily meets the criteria for a YES resolution. It even outperforms GPT-4o, the successor to GPT-4 and OpenAI's current strongest low-latency model.

Source: https://arxiv.org/pdf/2412.19437, page 31

bought Ṁ500 YES1y

Jensen Huang (CEO of NVIDIA) said that with Blackwell GPUs, you could train GPT-4 with only about 4 MW of power consumed. Looks like even without algorithmic improvements we can get there.

1y

Approx 4x efficiency improvement from silicon alone, based on latest GPUs being announced now (specifically mi300x Vs a100).

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.
© Manifold Markets, Inc.TermsPrivacy