What score will o1-pro achieve on FrontierMath?

Basic

Ṁ642

Feb 1

20%

0-4%

33%

5%-9%

26%

10-19%

17%

20-49%

50+%

This market resolves as soon as OpenAI's o1-pro is tested on FrontierMath by a reputable source. Currently, no model achieves even 2%, as noted on page 9 of https://arxiv.org/pdf/2411.04872

I will not bet in this market.

An introduction to FrontierMath can be found here https://epoch.ai/frontiermath

Update 2025-16-01 (PST) (AI summary of creator comment): - Closing Time: Extended until the release of o3.
- Resolution Criteria:
- If o1-pro is tested on FrontierMath by a reputable source before the release of o3, the market resolves accordingly.
- If no benchmark results by the release of o3, the market resolves as N/A.

This question is managed and resolved by Manifold.

Get

1,000

and

3.00

3 Comments

7 Holders

34 Trades

Sort by:

filled a Ṁ75 NO at 18% order

OpenAI claims 5.8% pass @1 for o1: https://openai.com/index/openai-o3-mini/

Those pass@4 imply this might be 10% or so.

@Usaar33 That's not precise enough for me to resolve. Unless something changes in the next few hours, I plan to resolve the market as N/A.

Since there are no FrontierMath results for o1-pro, I've extended the closing time until o3 is presumably released. If there are no benchmark results by then, it's very unlikely there will be any, as OpenAI won't spend resources testing an obsolete model, and this market will resolve as N/A.

Related questions

Related questions