Will Gemini achieve a score above 90% on the MATH benchmark? | Manifold

Will Gemini achieve a score above 90% on the MATH benchmark?

Plus

20

Ṁ4926

resolved Sep 16

Resolved

YES

1D

1W

1M

ALL

The current SOTA is 84.3% from GPT-4 Code Interpreter. Code & tool use is allowed.

This question is managed and resolved by Manifold.

Get

1,000

and

3.00

Sort by:

I should have specified the exact model. What I intended was the first Gemini 1.0 family, not the entire Gemini series. My bad guys. Since the question itself can be interpreted as the Gemini series, so I just resolve this to Yes.

Since this market has no restrictions on public availability or zero shot, I think this should probably already resolve as yes per Gemini 1.5 report

This is a separate MATH than the one that Google reported the benchmark on. And I don't see it beating GPT4 by so much, given most of the other scores were very close.

limit order for yes 10

Related questions

Will an AI score over 80% on FrontierMath Benchmark in 2025

+7% 1d23% chance

Before February 2025, will a Gemini model exceed Claude 3.5 Sonnet 10/22's Global Average score on Simple Bench?

Will Gemini outperform GPT-4 at mathematical theorem-proving?

Will Al achieve 95% or higher score on the FrontierMath benchmark before 2030?

Will Gemini-1.5-Pro-Exp-0801 Score Above 1165 in Scale AI's Coding Evaluation

Will Gemini exceed the performance of GPT-4 on the 2022 AMC 10 and AMC 12 exams?

By when will AI score >= 80% on FrontierMath

Will Gemini-1.5-Pro-Exp-0801 Score Above 1165 in Scale AI's Math Evaluation

Before what year will Al achieve 85% or higher score on the FrontierMath benchmark?

Will Gemini-1.5-Pro-Exp-0801 Score Above 90.35 (current #1) in Scale AI's Instruction Following Evaluation

Related questions

Will an AI score over 80% on FrontierMath Benchmark in 2025

Will Gemini exceed the performance of GPT-4 on the 2022 AMC 10 and AMC 12 exams?

Before February 2025, will a Gemini model exceed Claude 3.5 Sonnet 10/22's Global Average score on Simple Bench?

By when will AI score >= 80% on FrontierMath

Will Gemini outperform GPT-4 at mathematical theorem-proving?

Will Gemini-1.5-Pro-Exp-0801 Score Above 1165 in Scale AI's Math Evaluation

Will Al achieve 95% or higher score on the FrontierMath benchmark before 2030?

Before what year will Al achieve 85% or higher score on the FrontierMath benchmark?

Will Gemini-1.5-Pro-Exp-0801 Score Above 1165 in Scale AI's Coding Evaluation

Will Gemini-1.5-Pro-Exp-0801 Score Above 90.35 (current #1) in Scale AI's Instruction Following Evaluation

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules