[Carlini questions] Cost of a million output words in 2030 for an LLM that achieves at least current benchmark SOTA
4
200Ṁ579
2030
0.3 $
expected
92%
Below $0.10
3%
$0.10 - $0.49
1.3%
$0.50 - $0.99
1.3%
$1.00 - $2.99
1%
$3.00 - $5.99
0.8%
$6.00 - $9.99
0.8%
$10.00+

Full question: "An AI system that benchmarks at least as well as current state-of-the-art systems costs X$ to write a million output words (~10 books)"

Resolution Criteria:

Current state-of-the-art models include, e.g., GPT-4o, Claude-3.5 Sonnet, and Gemini 1.5 Pro. To compute cost-per-word, I will consider (a) for open source models, the cost of inference on a public cloud provider, or (b) for models available over APIs, whatever the cost is to use the API. I will only include models that are generally available to the public. "Benchmarks at least as well" means across a diverse range of benchmarks, the AI system achieves an average score at least as high as one of these two models as they exist today.

Motivation and Context:

Over the past two years, the cost-per-word to reach the original GPT-4 levels of performance has dropped from roughly $150 per million words to roughly $15 per million words today. Will this trend continue?

Question copied from: https://nicholas.carlini.com/writing/2024/forecasting-ai-future.html

Get
Ṁ1,000
to start trading!
Sort by:
opened a Ṁ600 YES at 85% order

wish the buckets were different. it's obviously going to be the first one. Gemini 2 Flash is already only $0.40 per million output tokens

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules