In 2024, will METR or Google announce the results of a METR eval on a Google LLM?
Basic
7
Ṁ189Jan 1
72%
chance
1D
1W
1M
ALL
METR = formerly ARC Evals (https://metr.org/)
if METR/Google reorgs and has a clear successor org, that org also applies for the purpose of this market
central YES cases:
if Google releases a model card with something like Gemini Supermega 2024 edition with METR exfiltration eval results, like OpenAI did for GPT4 technical report
does not have to be the specific exfiltration eval.
does not have to be included in initial model release paper. does not have to be specifically in a paper.
does not have to be any specific eval granularity. "METR ran the eval and it was all OK" would be ... annoyingly vague from whoever would write it, but it would count.
has to be confirmed-ish by Google and/or METR. can't be just a Twitter rumor.
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will Google announce that it's going to power Google Translate with an LLM-only based system (before 2024 end)?
36% chance
Will Google have a better LLM than OpenAI by 2025?
33% chance
Will there be a clear way to integrate LLMs with ads by the end of 2024?
36% chance
Will the new LLM released by Meta be open-source?
72% chance
Will Google cancel an LLM-based product by end of 2025?
59% chance
Will Meta AI's MEGABYTE architecture be used in the next-gen LLMs?
42% chance
Will the market about "Google mostly catching up to OpenAI in LLM quality by the end of 2024" resolve N/A?
15% chance
Will we see improvements in the TruthfulQA LLM benchmark in 2024?
74% chance
Will Europe be competitive in the LLM race compared to OpenAI or Google at the end of 2024?
6% chance
[Metaculus] Will Google implement a feature to explain targeted Google Ads before 2026?
50% chance