How will people run LLaMa 3 405B locally by 2025? | Manifold

How will people run LLaMa 3 405B locally by 2025?

11

565Ṁ317

Jan 1

91%

Gaming GPUs + heavy quantization (e.g. 6x4090 @ Q2_0)

65%

Unified memory (e.g. Apple M4 Ultra)

60%

Tensor GPUs + modest quantization (e.g. 4xA100 2U rackmount)

60%

Distributed across clustered machines (e.g. Petals)

41%

Server CPU (e.g. AMD EPYC with 512TB DDR5)

"Cloud" is a boring answer. User base of interest is somewhere between hobbyists with a budget and companies with a couple of self-hosted racks.

Get

1,000

to start trading!

People are also trading

Will RL work for LLMs "spill over" to the rest of RL by 2026?

Will CARB Proposed In-Use Locomotive Regulation be shown to cause freight/passenger diversion to vehicles before by 2030

Will the SF to LA segment of the California HSR be completed by 2050?

What will Manifolders mostly use LLMs for, by EOY 2025?

Will ULA have reused engines by the end of 2030?

Will it be possible for me to travel from San Francisco to Lighthaven in a Waymo in 2025?

Will the SF to LA segment of the HSR be completed by 2040?

Will the California HSR meet its Bay Area<->LA ridership goals by 2030 (8.5M/y)

MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?

Will cities have automated underground logistics networks before 2050?

Related questions

Will RL work for LLMs "spill over" to the rest of RL by 2026?

Will CARB Proposed In-Use Locomotive Regulation be shown to cause freight/passenger diversion to vehicles before by 2030

Will the SF to LA segment of the California HSR be completed by 2050?

What will Manifolders mostly use LLMs for, by EOY 2025?

Will ULA have reused engines by the end of 2030?

Will it be possible for me to travel from San Francisco to Lighthaven in a Waymo in 2025?

Will the SF to LA segment of the HSR be completed by 2040?

Will the California HSR meet its Bay Area<->LA ridership goals by 2030 (8.5M/y)

MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?

Will cities have automated underground logistics networks before 2050?

© Manifold Markets, Inc.•Terms•Privacy