What will be the best result an LLM based model gets in the 2025 IMO under fair conditions?
17
1kแน€4074
Jul 18
6%
Perfect score
13%
(non-perfect) Gold medal
29%
Silver medal
23%
Bronze medal
18%
Honorable mention
11%
No medal

Will resolve approx 1 week after the IMO problems are publicly released.
The model must have as input only the english+latex version of the problems.
Must not be a proof-seach specific model (aka not alpha-proof). This is a pseudo-benchmark for how good General AIs are at creative math. O3 and such are fair play
Ai must have same time constraints as competitors but no compute constraints, obviously no online access.

Competition Math
AI Benchmarks
IMO Grand Challenge
LLMs
IMO
Get
แน€1,000
to start trading!

People are also trading

Sort by:

Who will perform the evaluation within 1 week of IMO?

You realize grading solutions takes time of professional mathematicians/math olympians? And since there are ~5 candidate LLMs with different computer budgets, i think this question has no chance to be resolved objectively.

@mathvc LLM as a judge? I believe that they release grading guides (I might be totally wrong)

@AdamCzene at least as of April 2025 LLMs are not a reliable verifier of correct/wrong solution in informal language. They mostly output vibes instead of real verification

@mathvc matharena will probably do it

@DottedCalculator matharena oly rubric:

@jatloe it's not terrible

@DottedCalculator WHAT

People are also trading

ยฉ Manifold Markets, Inc.โ€ขTerms + Mana-only Termsโ€ขPrivacyโ€ขRules