What organization will have the top language model on LMSys overall leaderboards December 1st 2024?
โž•
Plus
46
แน€22k
Dec 3
18%
Google / Gemini
61%
OpenAI / ChatGPT
11%
Anthropic / Claude
3%
Meta / Facebook / LLama
4%
xAI / Grok / Elon empire
3%
Other

As of mid August, Google's Gemini leads LMSys pretty comfortably. Although its score has dipped below 1300.

https://chat.lmsys.org/?leaderboard

Note two things.

LMSys has statistical ties. If multiple orgs tie for first... we will split the payouts. although this is quite unlikely

Secondly, note that the update shows as "Last updated: 2024-08-06." even though it's already August 13th. We will go by the update date. So we are looking at the first snapshot on or after 2024-12-01.

Google seems like a substantial favorite. But maybe not, if GPT-5 / Strawberry ships.

Get แน€1,000 play money
Sort by:
bought แน€444 Other NO

Added xAI / Grok option.

They just released a preview for Grok 2.0
https://x.ai/blog/grok-2

Claiming "tied for 3rd / 4th" so far with large error bars.

Grok 2.0 won't win but they can be a contended with an improved model!

bought แน€444 OpenAI / ChatGPT YES

Interesting! The latest GPT-4o checkpoint... released 08/12 hits the number one spot.

By a big ELO margin over Gemini no less...

Lots of people online are grumbling... but it does go to show that, at the very least, OpenAI is pretty good at optimizing for the test.