Will the "so-called" Llama 3 actually be Llama 4? (Dense-->MoE)

#AI #LLMs #Meta (Facebook)

276

Ṁ838

Ṁ270

2026

10%

chance

ALL

I have no inside information but a wild guess:

Llama 3 WAS gonna be a dense model.

Yet, with the success release of Mixtral and Deepseek MoE, the whole team completely changed their model design, abandoned the model that is about to train, in progress, or even finished pretrained, and instead followed the new MoE approach.

This market will bet on my speculations. I will try to make the resolution rule as clear as possible, but since it will rely on some sort of inside information, some ambiguity will inevitably remain.

If the next-gen model from Meta is named differently, then it is considered Llama 3 if:

It has text language understanding/generation capability
It show (statistically) significant improvement over Llama 2 on LMSYS Arena Leaderboard (3* max(std dv of llama2-70b-chat, std dv of new model's strongest version))

Some Certain cases:

If Llama 3 is not MoE, the market will resolve to "NO"
Else, if anyone in meta stated that they was gonna do dense training, but changed idea, the market will resolve to "YES"
If two independent news sources confirmed my story, resolve to "YES"

I am setting a closing time of EOY 2025, since I believe Llama 3 will be released before this time. No release before that will resolve "NA".

In all the other cases, I will be very cautious on deciding the resolution based on public and expert opinions, some inside news, etc. Keep that in mind when you bet.

Note that another possibility is that Meta no longer releases the weight for Llama 3, and does not tell people if they used MoE.

If I cannot figure it out by March 01, 2026, I will resolve the market to "NA"

Get Ṁ200 play money

Related questions

Will the 400B+ open source Llama 3 model rank higher than GPT-4-Turbo-2024-04-09 on the lmsys leaderboard?

53% chance

Will Llama-4 be (open sourced and) as good as GPT-4?

74% chance

Will LLaMA-3 be on par with or better than GPT-4?

85% chance

Will Llama 3 use Mixture of Experts?

10% chance

Will Llama-3 (or next open Meta model) be obviously good in its first-order effects on the world?

81% chance

Will Llama 3 405B be open sourced

89% chance

Will Llama 3 400B be better than GPT-4?

50% chance

What month will Llama 3 400B+ be released?

How many active parameters will the largest Llama 3 have?

75% chance

Will Llama-3 be multimodal?

Sort by:

Not gonna resolve until we know if this 400B+ is out.

The market will resolve YES if this 400B+ is MoE, otherwise NO

If both MoE and Dense model were released under the name Llama 3, I am leaning towards resolving to YES as long as some report/meta ppl says that MoE started LATER than the dense model started.

@HanchiSun If they started the dense and MoE at the same time, then resolve to NO

@HanchiSun Arbitrage opportunity

@HanchiSun Don’t u guys want to make free money? 46<74, yet the latter is a subset of the previous

@HanchiSun I am not conditioning on Llama 3 being MoE, but requiring it for the market to resolve yes

How do you verify if they changed from dense to moe bc of the recent moe releases? Or does the reason matter at all? I am guessing from news sources as well?