6 months from now will I judge that LLMs had already peaked by Nov 2024?

1kṀ3407

resolved May 21

Resolved

ALL

I will compare the state of the art LLMs in June 2025 with current LLMs. I will resolve YES if there's not a marked improvement, no credible, verifiable improvement on the horizon and my judgement is that LLMs 'peaked' around end of 2024. I will resolve NO if something like GPT-5 comes out and it's a (subjectively) step improvement from GPT-4. Improvements around the edges and stuff like multimodality don't count. If Google releases something new that catches up with Claude 3.5/GPT-4 that doesn't count.

I use LLMs extensively daily and my friends have described me as having "such great judgment" but this market is still vibes-based at the end of the day so caveat emptor. I will answer any/all relevant clarifying questions in the comments and add stuff to the description if it helps clear things up. While most of the resolution will be mine, I will allow some external consensus to influence me

I will not trade in the market to avoid (appearance of) conflicts of interest

Update 2025-05-20 (PST) (AI summary of creator comment): The creator has indicated they are about to resolve this market NO, earlier than the originally stated timeframe. See the linked comment for their reasoning.

Technology

LLMs

Get

1,000

to start trading!

🏅 Top traders

#	Name	Total profit
1		Ṁ122
2		Ṁ121
3		Ṁ108
4		Ṁ50
5		Ṁ48

People are also trading

Will we have a popular LLM fine-tuned on people's personal texts by June 1, 2025?

1% chance

Will LLMs become a ubiquitous part of everyday life by June 2026?

82% chance

Will LLMs mostly overcome the Reversal Curse by the end of 2025?

72% chance

Will there be major breakthrough in LLM Continual Learning before 2026?

25% chance

Will Apple release its own LLM on par with state of the art LLMs before 2026?

7% chance

What will be true of Anthropic's best LLM by EOY 2025?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

50% chance

Will LLMs be the best reasoning models on these dates?

What organization will top the LLM leaderboards on LMArena at end of 2025? 🤖📊

Will China have the best LLM by the end of 2025?

Sort by:

was going to resolve this NO at market close but don't see any need to wait after recent spate of releases and google releasing gemini diffusion

@dlin007 are they good yet?

@dlin007 This doesn't make sense. If you think you can predict what models will be like in a month, then place a bet. But don't resolve early. Someone who claimed a peak right before the first reasoning models came out would look pretty foolish. There could be similar jumps in the near future.

(honestly, probably there won't be. still too soon to call)

@CraigDemel wait, wdym? i'm not claiming models have peaked now, just that they're already demonstrably (and subjectively) better than what was around in Nov 2024. do you disagree?

@TiredCliche still mid, but getting better

@dlin007 Apologies, I'm so embarrassed. I didn't remember how I had bet, and somehow perceived that I'd bet no and you'd resolved yes.

@CraigDemel I forgive you

@TiredCliche i forgive him too, but only because you did it first

@dlin007 Aw, thanks!

Do you have a working definition of "LLM" for these purposes? I see agent models gaining quickly on traditional LLMs, so a lot depends on which future models are considered LLMs, versus more complex systems of which an LLM is only one component.

is this gonna be resolved in June 2026 (18 months from now) or 6 months from now (June 2025)? its a no either way but clarification would be nice