
Is the LMSYS chatbot arena leaderboard trustworthy?
23
10káš11k2027
49%
chance
1H
6H
1D
1W
1M
ALL
LLMs can distinguish their own output from the output of different LLMs and they have a preference for their own output, so it's technically feasible to manipulate the leaderboard by throwing an LLM at the chatbot arena to upvote its own completions.
Has this happened yet? Will it happen soon?
Resolves NO iff, before 2027/7/1, credible media reports state that the lmsys leaderboard has been manipulated with sockpuppet accounts / fraudulent voting. A statement coming directly from lmsys would also count.
Resolves YES otherwise.
This question is managed and resolved by Manifold.
Get
1,000 to start trading!
People are also trading
Related questions
Which company has best AI model end of July? (Chatbot Arena Leaderboard)
What will be GPT-5's score on LMSYS Chatbot Arena?
Which company has best AI model end of June? (Chatbot Arena Leaderboard)
Which company has best AI model end of June? (Chatbot Arena Leaderboard)
Will the LMSYS Chatbot Arena still be 'a thing' in 2027, under the same evaluation method?
36% chance
Who will ever rank #1 in LMSYS Chatbot Arena Leaderboard in 2025?
Who will ever rank Top 10 in LMSYS Chatbot Arena Leaderboard in 2025?
Will GPT-5 top the LLMSys Chatbot Arena leaderboard within a month of its release?
73% chance
Which company has best AI model end of July? (Chatbot Arena Leaderboard)
Will a chatbot from a Chinese company top the LMSYS leaderboard in 2025?
29% chance