Resolution criteria

• By the end of the year 2025 (anywhere in the world) resolve “Yes” to the organization shown in the “Organization” column for the model ranked #1 by Arena Elo on LMArena’s text Chatbot Arena leaderboard when filtered to open‑weight models (“Open”/open weights). Use the page ordering at the timestamp. All other answers resolve “No.”

  • Primary source: https://lmarena.ai/leaderboard/text

  • Definition of “open”: open weights per LMArena policy; if an explicit “Open” filter is unavailable, treat any model whose License is not “Proprietary” as eligible. (newblog.lmarena.ai)

• If lmarena is moved to a different url or is renamed, the market shall refer to the successor.

• Ties/near‑ties: follow the page’s displayed rank at the cutoff time; if two entries have the same rank, resolve based on Elo scores. If scores are also identical, resolve to the entry with the greater vote count. If vote count is also identical, resolve to whichever model was released earlier.

Background

• LMArena (formerly “Chatbot Arena”) ranks LLMs via blind, head‑to‑head user votes and computes Arena Elo/Bradley–Terry ratings; it lists publicly released models (open weights, public APIs, or public services) and distinguishes first‑party vs third‑party endpoints. (news.lmarena.ai)

• The leaderboard and its data pipeline are maintained by the LMSYS/UC Berkeley team and mirrored on Hugging Face for stability. (news.lmarena.ai)

• Example contenders with open‑weight families (not exhaustive):

  • Meta (Llama), Alibaba (Qwen), Mistral AI (Mistral/Mixtral), DeepSeek (DeepSeek‑V), Databricks (DBRX), NVIDIA (Nemotron), AI2 (OLMo), TII (Falcon), Cohere For AI (Aya), Snowflake (Arctic), Microsoft (Phi), Apple (OpenELM), IBM (Granite), Zhipu AI (GLM). Eligibility depends on open‑weight availability and leaderboard listing per policy. (newblog.lmarena.ai)

Considerations

• Pre‑release/anonymous models sometimes battle on LMArena but only publicly released models appear on the leaderboard; third‑party endpoints are labeled. Only open‑weight entries count here. (newblog.lmarena.ai)

• LMArena also runs separate multimodal/vision leaderboards; this market resolves on the text Chatbot Arena leaderboard only. (oldblog.lmarena.ai)

• Models can be deprecated if no longer publicly accessible; rankings update continuously as votes accrue, so the end‑of‑year snapshot governs resolution. (newblog.lmarena.ai)