As of mid August, Google's Gemini leads LMSys pretty comfortably. Although its score has dipped below 1300.
https://chat.lmsys.org/?leaderboard
Note two things.
LMSys has statistical ties. If multiple orgs tie for first... we will split the payouts. although this is quite unlikely
Secondly, note that the update shows as "Last updated: 2024-08-06." even though it's already August 13th. We will go by the update date. So we are looking at the first snapshot on or after 2024-12-01.
Google seems like a substantial favorite. But maybe not, if GPT-5 / Strawberry ships.
1,000
3.00Google / Gemini enters number one...
https://x.com/lmarena_ai/status/1857110672565494098
With a pretty pedestrian 1344 ELO.
I don't know if returns to scale are diminishing... but looking less and less likely we see a breakthrough model before Thanxgiving
Added xAI / Grok option.
They just released a preview for Grok 2.0
https://x.ai/blog/grok-2
Claiming "tied for 3rd / 4th" so far with large error bars.
Grok 2.0 won't win but they can be a contended with an improved model!
