In 2024, will METR or Google announce the results of a METR eval on a Google LLM?
Basic
7
Ṁ189Jan 1
72%
chance
1D
1W
1M
ALL
METR = formerly ARC Evals (https://metr.org/)
if METR/Google reorgs and has a clear successor org, that org also applies for the purpose of this market
central YES cases:
if Google releases a model card with something like Gemini Supermega 2024 edition with METR exfiltration eval results, like OpenAI did for GPT4 technical report
does not have to be the specific exfiltration eval.
does not have to be included in initial model release paper. does not have to be specifically in a paper.
does not have to be any specific eval granularity. "METR ran the eval and it was all OK" would be ... annoyingly vague from whoever would write it, but it would count.
has to be confirmed-ish by Google and/or METR. can't be just a Twitter rumor.
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will Google have a better LLM than OpenAI by 2025?
31% chance
Will Google announce that it's going to power Google Translate with an LLM-only based system (before 2024 end)?
57% chance
Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?
57% chance
Will Google cancel an LLM-based product by end of 2025?
59% chance
Will the new LLM released by Meta be open-source?
61% chance
Will there be a clear way to integrate LLMs with ads by the end of 2024?
78% chance
Will Meta release an LLM named something like 'Llama 3.3' by the end of 2024?
50% chance
Will Meta AI's MEGABYTE architecture be used in the next-gen LLMs?
45% chance
will top LLM models use formal methods by EOY 2024?
24% chance
[Metaculus] Will Google implement a feature to explain targeted Google Ads before 2026?
50% chance