Gemini Ultra was released on Feb 8, 2024
As of Feb 9, 2024, it is expected to achieve a higher Elo than GPT-4 according to this market
As of Feb 9, 2024, GPT-4 has an Elo of 1253
Resolution criteria:
Resolves YES if some version of Gemini Ultra achieves an Elo of 1300 or higher on Chatbot Arena at any point before May 1, 2024
Resolves NO if Gemini Ultra does not reach an Elo of 1300, or if it never appears on the leaderboard before the deadline
Resolves N/A if the leaderboard is no longer publicly accessible at the time of closing
Related questions
@Sss19971997 By "Gemini Ultra", I am referring to the model that powers Gemini Advanced. You might be thinking of another model, Gemini Pro, that powers Gemini. Gemini Pro shows up on the leaderboard as "Bard (Gemini Pro)", "Gemini Pro (Dev API)", and "Gemini Pro". None of these count as "Gemini Ultra".
@howahlah When I mentioned Gemini Pro, I was making an analogy. The Bard(Pro) is not an LLM but a search-engine augmented chatbot. So I was asking if Gemini Advanced (Gemini Ultra augmented with search) shows up on the leader board, does it count as Gemini Ultra?
@Sss19971997 Yes, I would consider Gemini Advanced "some version of Gemini Ultra". This is analogous to Gemini Pro showing up as "Bard (Gemini Pro)".
@Sss19971997 sure, feel free to bet up. Bear in mind that if it doesn't appear on the leaderboard by the deadline this will resolve NO as well.