What Elo will gpt2-chatbot have on the LMSYS leaderboard?

#AI #Technology #OpenAI #ChatGPT #GPT-5

276

Ṁ1.6k

Ṁ1k

Jul 1

ALL

1.1%

Below 1240 (not SOTA)

1.2%

1241-1270 (match GPT4/Claude 3)

97%

1271-1350 (new SOTA)

1351+ (generation defining - as big a gap as GPT4 v GPT3.5)

Question resolves to its Elo one week after it first appears on the leaderboard.

If [OpenAI / an employee there / whatever org created it] acknowledges this model and gives it a new name (eg, GPT4.5), will resolve to that model's Elo.

If no gpt2-chatbot model is listed on the leaderboard by June 30, will resolve N/A.

https://chat.lmsys.org/?leaderboard

Get Ṁ600 play money

Related questions

Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?

25% chance

Which companies will achieve a higher ELO rating than OpenAI on ChatBot Arena in 2024? ( >= week)

What organization(s) will be ranked #1 in the LMSYS Org Chatbot Arena Leaderboard at the end of June, 2024?

What will be the highest ELO on Chatbot Arena on July 1, 2024?

1298

Will GPT 4.5/GPT 5 have 30 more elo on the LMSYS leaderboard a two weeks after it is released.

77% chance

What organization will have the highest ELO score in the LMSYS Org Chatbot Arena Leaderboard at the end of June, 2024?

Will GPT-4-Turbo be ranked in the top 20 on the Chatbot Arena Leaderboard at the end of 2025?

39% chance

What organization(s) will be ranked #1 in the LMSYS Org Chatbot Arena Leaderboard at the end of September, 2024?

Will a model from Keen Technologies by John Carmack be in the top 10 on the LMSYS leaderboards by the end of 2024?

23% chance

Will the GPT4+code-interpreter+search score > 1350 on Lmsys Arena Leaderboard?

Sort by:

GPT-4o is on the leaderboard and per the resolution criteria, I will wait one week for the Elo to stabilize and then resolve on Thursday May 23.

Based on this tweet, expecting to resolve this to the GPT-4o Elo

https://x.com/liamfedus/status/1790064963966370209

That tweet greatly exaggerated the final ELO

@Uaaar33 Seriously. Though the actual Elo is way closer to my subjective experience (small improvement)

Looks like 2 new potential successors were released "im-a-good-gpt2-chatbot" and "im-also-a-good-gpt2-chatbot". If these both appear on the leaderboard, I'm inclined to use the highest rated Elo to resolve this question.

Even if unacknowledged by OpenAI, I'll count these. (resolves N/A if no successor is released)

Let me know if any concerns with that.

gpt2-chatbot was just removed from the list of Direct Models a few minutes ago. I was having a conversation with it and it gave me an error saying there was too much traffic, then was no longer an option to select when I refreshed.