Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers? | Manifold

Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?

8

190Ṁ142

2050

67%

chance

1H

6H

1D

1W

1M

ALL

This question is meant to measure the gap between solving the main math-based benchmarks at the time of market creation, and applying mathematics in the real world.

Data science / data analysis / statistics positions: I'll accept anything in this general realm of jobs, with the caveat that I won't require any significant coding. For instance an AI working as an ML scientist would resolve this market YES, but this is not a requirement.

There is no requirement that any human entry-level workers be fired/replaced, e.g. if for some reason both humans and AI are employed to do the same work that counts.

Technical AI Timelines

Get

1,000

to start trading!

People are also trading

Will any AI model achieve > 40% on Frontier Math before 2026?

-6% 1d68% chance

Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?

Benchmark Gap #4: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, how many months will it be before an AI is listed as a (co) first author on a published math paper?

Benchmark Gap #3: Once a model achieves superhuman performance on a competitive programming benchmark, will it be less than 2 years before there are "entry level" AI programmers in industry use?

Once a model achieves superhuman performance on a competitive programming benchmark, will it be less than 5 years before there are "entry level" AI programmers in industry use?

Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?

Will models be able to do the work of an AI researcher/engineer before 2027?

Benchmark Gap #1: Once we have a language model that achieves expert human performance on all *current* major NLP benchmarks, how many years will it be before we have an AI with human-level language skills?

Will an AI score over 80% on FrontierMath Benchmark in 2025

Will an AI model outperform 95% of Manifold users on accuracy before 2026?

Related questions

Will any AI model achieve > 40% on Frontier Math before 2026?

Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?

Benchmark Gap #4: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, how many months will it be before an AI is listed as a (co) first author on a published math paper?

Benchmark Gap #3: Once a model achieves superhuman performance on a competitive programming benchmark, will it be less than 2 years before there are "entry level" AI programmers in industry use?

Once a model achieves superhuman performance on a competitive programming benchmark, will it be less than 5 years before there are "entry level" AI programmers in industry use?

Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?

Will models be able to do the work of an AI researcher/engineer before 2027?

Benchmark Gap #1: Once we have a language model that achieves expert human performance on all *current* major NLP benchmarks, how many years will it be before we have an AI with human-level language skills?

Will an AI score over 80% on FrontierMath Benchmark in 2025

Will an AI model outperform 95% of Manifold users on accuracy before 2026?

© Manifold Markets, Inc.•Terms•Privacy