Will a reliable source reveal the existence of a zettascale computer before 2025?

Plus

Ṁ19k

Jan 2

chance

ALL

The advancement of computational power has been guided by various metrics, including FLOPS (Floating Point Operations per Second). Zettascale computing represents a major leap, equivalent to 10^21 FLOPS. It goes well beyond the current exascale systems, opening new horizons in fields like climate modeling, medical research, and artificial intelligence. Minifloat precision refers to the use of reduced precision floating-point numbers, which can be a trade-off between computation speed and accuracy. Having at least 8 bits for the minifloat precision can help ensure that the system has a reasonable level of accuracy while still enabling high-speed computation.

Will reliable sources reveal the existence of a zettascale computer with a minifloat precision of at least 8 bits before January 1, 2025?

Resolution Criteria:

This question will resolve to "YES" if, before January 1, 2025, a zettascale computer meeting the following criteria is revealed by reliable sources:

Computational Power: The computer should be capable of achieving zettascale computing, equivalent to 10^21 FLOP/s or higher. This will refer to theoretical peak performance FLOP. Reported sparse FLOP will not count.
Contiguous computer: the computer must be a single contiguous supercomputer, for example inside of a single data center, rather than being a set of computers loosely connected across space or time.
Minifloat Precision: The FLOP/s will be measured in minifloat precision of at least 8 bits, allowing for a balance between computational speed and accuracy. For example, FP8 and FP16 are acceptable binary interchange formats to use for resolution.
Independent Revelation: The computer’s capabilities must be revealed by reliable source, such as statements from a major tech firm, reporting from knowledgable journalists, or highly reputable third party bloggers. If there is credible reason to dispute the veracity of a claim about a particular computing system, this question will wait until more evidence has come to light before resolving.

I will use my discretion when resolving this question, possibly in consultation with experts in the field of high-performance computing, to ensure that the criteria are met and that the zettascale computer with at least 8 bits of minifloat precision has indeed been revealed by a reliable source.

This question is managed and resolved by Manifold.

#️ Technology

#Artificial Intelligence

#Computing

Get

1,000

and

3.00

11 Comments

32 Holders

113 Trades

Sort by:

@MatthewBarnett Does the reveal have to happen within 2025, or just the construction of the supercomputer?

@YafahEdelman It must be revealed before January 1st 2025.

Why is this tagged quantum computing?

https://manifold.markets/MatthewBarnett/will-an-entity-be-confirmed-to-have

Will an entity be confirmed to have purchased at least 300,000 H100 GPUs before January 1st, 2025?

27% chance. The demand for high-performance computing hardware, particularly NVIDIA's H100 Tensor Core GPU, has seen a surge with the rise of advanced artificial intelligence applications. Major investments, like the one seen with Inflection AI which purchased 22,000 H100 GPUs, emphasize the increasing need for these GPUs in large-scale AI projects. Will any entity, anywhere in the world, be confirmed to have placed an order for a cumulative total of at least 300,000 NVIDIA H100 Tensor Core GPUs by January 1st, 2025? Resolution Criteria: This question will resolve to "YES" if, before January 1st, 2025, credible evidence confirms that an entity, such as a government, corporation, non-profit, university, or single individual: Placed a valid order for a cumulative total of at least 300,000 NVIDIA H100 Tensor Core GPUs. Provided proof of this purchase intent through: a. Official statements or press releases from the entity itself, NVIDIA, or any other relevant stakeholder. b. Credible news reports or articles from well-established media outlets known for reliable reporting in the tech industry. c. Financial reports or documentation that disclose the intended purchase quantity of GPUs. Made this purchase intent for any purpose, including research, deployment in data centers, commercial product development, or any other use. Evidence of intent to purchase from secondary markets, i.e., intending to buy from a third party that is not NVIDIA, will also be considered valid as long as the entity's total intended GPU order reaches the 300,000 threshold. All cards labeled "H100" will count as an H100 GPU for the purpose of this question. Dual H100 NVL cards count as two H100 GPUs for the purpose of this question. I will use my discretion when resolving this question, possibly in consultation with experts, to ensure the criteria are met and that there is solid evidence of an entity's purchase for the specified number of GPUs.

@NoaNabeshima This should mean that this market resolves YES, right?

300k GPUs × 4k TeraFLOPs > 10^21 FLOPS

@Pykess Purchasing the GPUs isn't the same as making a giant computer with them

@Pykess I think ability to get (cheap?) power in one location might be a constraint (not confident)

@NoaNabeshima Who was the single entity that ordered the 300k GPUs? A redistributor? Or a compute-centric company like Microsoft/OpenAI/Department of Energy/etc?

@Pykess https://manifold.markets/MatthewBarnett/will-an-entity-be-confirmed-to-have#svYWp8l6kx7qAVK5NueC

@NoaNabeshima You'd want about 300MW for that many GPUs and some datacenters with that much exist now I believe. That being said the h100 only has 2k dense flops, not 4k, so 300k H100s doesn't get you to a zettaflop even if you wanted to use them all as one supercomputer.

(ThoughI think if someone gets to a zettaflop in 2024 it might instead be on the B100 which comes out later this year.)

I'm guessing H100s are the most likely way for a supercomputer to have lots of 8-bit FLOP/S before 2025.

An H100 SXM gets 4k teraFLOP/S or 15.6 OOM of FLOP/S. [0]
Zettascale is 21 OOM. So there's 5.4 OOM to cover.

An H100 is $30K. So without discount from Nvidia it′s $10B to reach zettascale. That's about the size of OpenAI's Microsoft investment. It would be a substantial chunk of the H100s Nvidia sells per year at current rates, like 20%? [1]

I think if it exists it's not that likely that it'll be revealed in a reliable way before 2025. I think the probability that it will exist is 35%. And revealed is 20%? Feels like the probabilities could move quickly with info.

Some other facts:
- Sometimes the top 1 supercomputer on TOP500 uses mostly Nvidia GPUs. The most A100s currently in a TOP500 supercomputer is Leonardo with 14K which is with high probability smaller than the largest private A100 supercomputer, at least going by rumours ab the computer to train GPT-4..
- TOP500 uses FP64 for FLOP.
- I think FLOP/S in TOPS-500 is always calculated with GPUs but I'm not sure.
- The current top-1 supercomputer cost $600M.
- The FLOP/S of the top-1 supercomputer increases by an OOM every ~4.3 years [2]

[0] - https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet
[1] - https://finance.yahoo.com/news/nvidia-sold-900-tons-h100-140613078.html
[2] - https://ourworldindata.org/grapher/supercomputer-power-flops

NVIDIA H100 Tensor Core GPU Datasheet

This datasheet details the performance and product specifications of the NVIDIA H100 Tensor Core GPU. It also explains the technological breakthroughs of the NVIDIA Hopper architecture.

Related questions

Related questions