How will people run DeepSeek Coder v2 236B locally by 2025?
How will people run DeepSeek Coder v2 236B locally by 2025?
50%
Lobotomy levels of quantization (e.g. Q2_K)
50%
Unified memory (e.g. M3 Ultra, Mac Studios)
50%
Non-GPU main memory (e.g. AMD EPYC with 512GB DDR5)
50%
Gaming GPUs in one motherboard (e.g. 4090s)
50%
Tall clustering (e.g. Mac Studios over Thunderbolt)
50%
Wide clustering (e.g. Petals)
50%
HBM FPGA/ASIC dark horse (e.g. AMD rains Versal chips like manna)

DeepSeek Coder v2 is arguably a frontier model from a stellar Chinese team. It has cutting edge efficiency tricks, improved reasoning thanks to extensive code pretraining, and what looks like an excellent math corpus.


It's also an absolute chonker despite being MoE.


What's going to be the path for GPU poors needing to run LLMs in "ITAR mode" without building their own private datacenter or buying NVIDIA racks?


Resolves to as many options that apply, based on public information. If there is no public information resolves based on vibes and a DIY pricing spreadsheet.

(Chinese overlords variant of Llama 3 405B question in case Meta gets cold feet.)

Get
Ṁ1,000
to start trading!


What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.
© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules