Resolves YES if any instance Llama 3 400B tops GPT-4 turbo (currently in 1st place) in the LMSYS Chatbot Arena Leaderboard within a year of its release.
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ545 | |
| 2 | Ṁ226 | |
| 3 | Ṁ139 | |
| 4 | Ṁ122 | |
| 5 | Ṁ101 |
GPT-4-Turbo-2024-04-09 is currently tied for 5th place on LMSYS ("rank" 4). I'm assuming this market resolves based on the top LMSYS model with "GPT-4-Turbo" in its name a year after today, i.e., July 5, 2025. I'm also assuming this resolves based on Llama-3.1-405B and not future Llama 3 versions.
I'd recommend changing the title from "Will Llama 3 400B be better than GPT-4?" to "Will Llama-3.5-405B beat GPT-4-Turbo?" FYI this market is also set to close Dec 31, but 3.1 405B is only out today.
Extrapolating the 8b to 70b ELO trend for LLAMA-2 and LLAMA-3 as log-linear to 400B, I get that 400B should be 5ish ELO below Turbo. I suppose that means Meta could throw some extra fine-tuning compute at 400B to get it above Turbo to look good, but I'm not sure how easy/worthwhile that is (especially if they barely beat Turbo).