Will a LLM trained with FP4 have competitive performance in 2 years time?
Will a LLM trained with FP4 have competitive performance in 2 years time?
16
1kṀ2955
resolved Mar 4
Resolved
NO

"Currently, the technology for 4-bit training does not exists, but research looks promising and I expect the first high performance FP4 Large Language Model (LLM) with competitive predictive performance to be trained in 1-2 years time." (see: https://timdettmers.com/2023/01/16/which-gpu-for-deep-learning/)

Granted, the model must be open source for us to know, so the market will resolve based on publicly available information.

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ158
2Ṁ152
3Ṁ90
4Ṁ55
5Ṁ13

People are also trading

Will a LLM trained with FP4 have frontier-level performance before 2028?
31% chance
Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?
50% chance
Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?
26% chance
Will a publicly-available LLM achieve gold on IMO before 2026?
31% chance
Will LLMs mostly overcome the Reversal Curse by the end of 2025?
72% chance
Will there be major breakthrough in LLM Continual Learning before 2026?
25% chance
Will China have the best LLM by the end of 2025?
17% chance
Will Apple release its own LLM on par with state of the art LLMs before 2026?
7% chance
Will a majority of Harvard undergrads train an LLM within 3 years as per Altman?
4% chance
Will LLM training costs fall 100x by 2028?
92% chance


Sort by:
4mo

@typedfemale Hi! Would you mind resolving this market?

3mo

@Gabrielle @Bayesian we got a lesson about this in Discord, here

My executive summary:

  • There's some evidence that FP4 is 'around the corner' and may demonstrate some of these qualities

But it's not enough to qualify for the market's criteria of:

  • Open source

  • Publicly available information

  • As of 21 January 2025

If someone disagrees with the way I'm spinning the summary of the conversation, post here!

3mo

@Eliza it should resolve NO

3mo

@PoliticalEconomyPK Well, I agree, but do we need to wait for a panel of 3 moderators to weigh in? I tried to wrangle some with no luck and it's been some weeks by now.

According to the linked analysis from sof, this simply did not happen. It doesn't sound like an ambiguity or a judgement call, but just "did anyone refute this analysis" (no).

So, let's resolve it no as "the resolution is obvious" rather than "ambiguous".

4mo

@Manifold

@mods

@ManifoldAI

Any AI expert can chime in and resolve this market? According to a prompt on chatGPT that I made, this should resolve no

predictedNO 2y

Exclusively in FP4? Or does partially in FP4 count. What if the model is on average 60% FP4 over the course of training?

2y

I guess you covered this with "trained in 4-bit (to some extent)"

predictedNO 2y

https://arxiv.org/pdf/2212.09720.pdf

predictedNO 2y

@NoaNabeshima This is ab post-training precision adjustments

2y

Competitive with what? SOTA with fp16?

predictedNO 2y

This seems important @typedfemale
Will this resolve YES if scaling laws suggest a 4-bit model would be competitive if compute-matched to a SOTA 16-bit model?

predictedNO 2y

(but there isn't a trained SOTA 4-bit model)

2y

@NoaNabeshima Yes, you need to be better than everything else, but be trained in 4-bit (to some extent)

2y

@typedfemale Finetuned w 4 bit would trigger Yes? 80% of parameters in 4 bit would trigger Yes?

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.

People are also trading

Will a LLM trained with FP4 have frontier-level performance before 2028?
31% chance
Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?
50% chance
Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?
26% chance
Will a publicly-available LLM achieve gold on IMO before 2026?
31% chance
Will LLMs mostly overcome the Reversal Curse by the end of 2025?
72% chance
Will there be major breakthrough in LLM Continual Learning before 2026?
25% chance
Will China have the best LLM by the end of 2025?
17% chance
Will Apple release its own LLM on par with state of the art LLMs before 2026?
7% chance
Will a majority of Harvard undergrads train an LLM within 3 years as per Altman?
4% chance
Will LLM training costs fall 100x by 2028?
92% chance
© Manifold Markets, Inc.•Terms•Privacy