What organization will have the top language model on LMSys overall leaderboards December 1st 2024?
💎
Premium
52
Ṁ47k
Dec 3
16%
Google / Gemini
75%
OpenAI / ChatGPT
4%
Anthropic / Claude
1.1%
Meta / Facebook / LLama
2%
xAI / Grok / Elon empire
1.4%
Other

As of mid August, Google's Gemini leads LMSys pretty comfortably. Although its score has dipped below 1300.

https://chat.lmsys.org/?leaderboard

Note two things.

LMSys has statistical ties. If multiple orgs tie for first... we will split the payouts. although this is quite unlikely

Secondly, note that the update shows as "Last updated: 2024-08-06." even though it's already August 13th. We will go by the update date. So we are looking at the first snapshot on or after 2024-12-01.

Google seems like a substantial favorite. But maybe not, if GPT-5 / Strawberry ships.

Get
Ṁ1,000
and
S3.00
Sort by:
bought Ṁ222 Google / Gemini YES

Google / Gemini enters number one...
https://x.com/lmarena_ai/status/1857110672565494098

With a pretty pedestrian 1344 ELO.

I don't know if returns to scale are diminishing... but looking less and less likely we see a breakthrough model before Thanxgiving

bought Ṁ444 Other NO

Added xAI / Grok option.

They just released a preview for Grok 2.0
https://x.ai/blog/grok-2

Claiming "tied for 3rd / 4th" so far with large error bars.

Grok 2.0 won't win but they can be a contended with an improved model!

bought Ṁ444 OpenAI / ChatGPT YES

Interesting! The latest GPT-4o checkpoint... released 08/12 hits the number one spot.

By a big ELO margin over Gemini no less...

Lots of people online are grumbling... but it does go to show that, at the very least, OpenAI is pretty good at optimizing for the test.

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules