How will this market resolve if Meta trains the best LLM, open sources it, and then Mistral finetunes it for a small to moderate amount of additional tokens which makes it perform somewhat better than the original Meta model at the end of 2024?

(As seems to be what's going on with mistral-medium.)

@RyanGreenblatt I think I’ll run a poll to see what people want. Or maybe I’ll resolve both as yes.

Related:

Best in what regard?

@mattyb Good question. I’m mostly thinking using this leaderboard: https://chat.lmsys.org/ (go to the leaderboard tab). If you have any other suggestions then let me know.

@AbhinavSrinivas Ok, by the Arena Elo leaderboard. Thanks