In 2024, will METR or Google announce the results of a METR eval on a Google LLM?
7
31
αΉ190αΉ170
2025
72%
chance
1D
1W
1M
ALL
METR = formerly ARC Evals (https://metr.org/)
if METR/Google reorgs and has a clear successor org, that org also applies for the purpose of this market
central YES cases:
if Google releases a model card with something like Gemini Supermega 2024 edition with METR exfiltration eval results, like OpenAI did for GPT4 technical report
does not have to be the specific exfiltration eval.
does not have to be included in initial model release paper. does not have to be specifically in a paper.
does not have to be any specific eval granularity. "METR ran the eval and it was all OK" would be ... annoyingly vague from whoever would write it, but it would count.
has to be confirmed-ish by Google and/or METR. can't be just a Twitter rumor.
Get αΉ200 play money
Related questions
Will Google have the best LLM by EOY 2024?
33% chance
Which company will have the best LLM by the end of 2024?
Will Google announce that it's going to power Google Translate with an LLM-only based system (before 2024 end)?
63% chance
Will Google, Amazon, Apple, or Samsung have their voice assistant integrated with an LLM, by 2024 end?
93% chance
Will there be an open source LLM as good as GPT4 by June 2024?
39% chance
Will Google cancel an LLM-based product by end of 2025?
59% chance
Will there be 20+ LLMs that match or outperform GPT-3.5's performance by the end of 2024?
89% chance
Will an open-source LLM beat or match GPT-4 by the end of 2024?
64% chance
Will Meta announce integration of a LLaMA model for businesses on Whatsapp, in 2024?
39% chance
Will Apple launch an LLM product in 2024?
86% chance