This will be based on whatever Meta calls Llama-4, whether or not it deserves that name, or if it renames its next larger LLM to not include 'llama' I will use best judgment on whether it counts. If Meta does not release a relevant model by EOY 2025 this resolves to NO. If the model is not open sourced, it does not count.
By default will judge based on the leaderboard here: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard
Once it has been on the leaderboard for 7 days if it is close to allow ratings to settle, or if the resolution is obvious in either direction for any reason, I will resolve. If I feel the leaderboard is clearly wrong or it is not available at the time and the answer is non-obvious, I will consult experts and/or use a Twitter poll.
Which GPT-4? The GPT-4 that's serving now is miles ahead on all of the benchmarks compared to what was originally released.
@AdamTreat https://github.com/facebookresearch/llama/blob/main/LICENSE
"2. Additional Commercial Terms. If, on the Llama 2 version release date, the
monthly active users of the products or services made available by or for Licensee,
or Licensee's affiliates, is greater than 700 million monthly active users in the
preceding calendar month, you must request a license from Meta, which Meta may
grant to you in its sole discretion, and you are not authorized to exercise any of the
rights under this Agreement unless or until Meta otherwise expressly grants you
such rights."
@ZviMowshowitz ok, then the question is do you have (or have any expectation of having) greater than 700 million monthly active users ;)