AI people are very excited for Meta's new Llama 405B model. It is rumored to be coming out on Tuesday July 23rd, and a copy possibly already leaked.
https://deepnewz.com/ai/meta-s-llama-3-405b-set-release-on-july-23-2024-new-feature-3-1-3-days-left
The rumors are that the model will be the first open source model "better than GPT-4."
https://deepnewz.com/ai/meta-s-llama-3-1-405b-model-leaked-on-4chan-ahead-july-23-launch-size-820-gb
Once the model is released, and has 2+ weeks to run on LMSys, where will it rank?
To be specific, we will check the first LMSys overall leaderboard table that's marked as on or after Monday August 12th. We may delay this if the model is released later than expected, but this will be updated in the rules if that is the unlikely case.
As usual with LMSys, we count statistical ties as reported on the leaderboards.
So a YES would result for a tie for first place or sole first place, presumably against the GPT-4o model that currently leads LMSys as of July 22nd.
https://chat.lmsys.org/?leaderboard
Claude 3.5 Sonnet currently ranks second, tied with Gemini...
1,000
3.00
Llama 3.1 405B -- debuts tied for #3 with Gemini.
https://x.com/lmsysorg/status/1818321701052276990
Based on only 6,000 votes -- but I would say pretty unlikely it will climb 20+ ELO points to get to GPT-4o from here.
The model is doing well on ScaleAI's secret metrics.
https://x.com/alexandr_wang/status/1815775286195331411
As good as Claude 3.5 Sonnet on some, and even better on others
