Background

From a recent arXiv preprint,

We introduce FrontierMath, a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians. The questions cover most major branches of modern mathematics -- from computationally intensive problems in number theory and real analysis to abstract questions in algebraic geometry and category theory. Solving a typical problem requires multiple hours of effort from a researcher in the relevant branch of mathematics, and for the upper end questions, multiple days. FrontierMath uses new, unpublished problems and automated verification to reliably evaluate models while minimizing risk of data contamination. Current state-of-the-art AI models solve under 2% of problems, revealing a vast gap between AI capabilities and the prowess of the mathematical community. As AI systems advance toward expert-level mathematical abilities, FrontierMath offers a rigorous testbed that quantifies their progress.

Resolution Criteria

This question resolves to the year bracket when a fully automated Al system achieves an average accuracy score of 95% or higher on the FrontierMath benchmark. If the year is 2041 and AI has not achieved that 95% score by that year then the question will be resolved to "Not Applicable."

• Verification: The score must be verified by credible sources such as peer-reviewed research papers, arXiv preprints, or independent evaluations from reputable Al research institutions.

• Autonomy: The Al must solve problems without any human intervention, external assistance, or reliance on pre-existing solution datasets.

• Compute Resources: There is no limitation on computational resources; Al systems can utilize unlimited resources to attempt solutions.