This market is centered on whether an open source Large Language Model (LLM) will achieve a higher Elo rating than OpenAI’s GPT-4 on the Chatbot Arena platform by the end of 2024.

Chatbot Arena utilizes a crowdsourced, randomized battle platform where user votes contribute to computing Elo ratings. This specific market will be resolved based on the Elo ratings of the models as reported by Chatbot Arena.
The link to the website: https://chat.lmsys.org/

The resolution will consider the highest Elo rating recorded for an open source LLM compared to GPT-4’s Elo rating as of December 31, 2024.

It’s important to note that only the Elo ratings will be used for determining the outcome, not considering other benchmarks like MT-Bench or MMLU scores.

The latest update of the leaderboard by December 20, 2024, will be used for the final assessment.