I (Tamay Besiroglu) bet the mathematician Daniel Litt that the best AI models in March of 2030 would be capable of generating math papers in Number Theory at the level of quality of papers published in Annals today (i.e. 2025). https://x.com/tamaybes/status/1899262088369106953?s=46
The AI would receive no detailed guidance relevant to the mathematics research, and is required to accomplish this task autonomously.
The AI system(s) are granted a total budget of $100k in inference compute per paper.
This bet would be resolved on the basis of Daniel Litt’s judgment.
Update 2025-03-21 (PST) (AI summary of creator comment): Novel Research Requirement Clarification:
For a YES resolution, the AI must perform novel research autonomously, not just produce a paper that could pass as research.
Update 2025-03-23 (PST): - Budget Currency: The $100k inference compute budget is expressed in nominal dollars (current currency) with no inflation adjustment. (AI summary of creator comment)
@LocalGlobal This feels to me a bit like "Those sea lions that were trained to build motorcycles a few years back can source parts and draft designs, sure, but their welding technique is still terrible!"
Without a deeper model of what drives AI progress, the phrase "currently models cannot" is just not predictive of anything.
@MaxMorehead Check your biases folks, Daniel Litt is neither dumb nor an AI skeptic.
FWIW I think his implied odds are too low too.
But the gap is wild!
@jgyou to be clear, I'm saying the market is too high, since even Tamay thinks this is (slightly) less than 50%.
@Bayesian This is a bit like saying in 1999 that people who have historically made money in the stock market in the age of the world wide web think that Internet is going to change everything. The Internet did change everything, but a lot of the stocks were overvalued.
Manifold hasn't existed for that long. I think strong prediction performance across many decorrelated domains (or over a long period of time) is a greater signal to me than doing well on Manifold. And while people who predicted the current state of AI deserve a lot of additional weight in their opinions, but I don't see any particularly strong meta-level reason to be convinced that the miscalibration of the past will continue in the future. So we are left to think mainly at the object-level at the state of AI science and the arguments for different AI timelines.
I do think that world-changing AI is quite possibly coming soonish, but probably not by 2030 (though I also think this market is a lot weaker than some "AGI" definitions).
@mathvc One of my looser bets, basically I expect math to kind of just be solved (to >human level) by reasoning models scaling up + modestly impactful new techniques by 2030. It's something that requires 0 real world interaction and can be verified post-facto as true/false.
No knowledge of Annals or modern frontier math research.
@DavidHiggs I am not so sure a paper that brings a sufficiently new contribution can be easily verified post-factum, unless the AI model manages to translate its proofs to formal language that can be checked automatically. Checking Wiles' proof of Fermat's last theorem took years
@mariopasquato I’d half expect formal proof to be the default, and human readable to be the required translation, but haven’t really thought about it deeply.
@DavidHiggs I think this us a misunderstanding of what mathematicians do. Proof is not the main product of a math paper, rather it is human understanding of math (with proof as important tool/constraint). Historically, proofs that were valid, but didn't increase human understanding (e.g. 4 color theorem) were viewed as something between underwhelming and almost useless.
@AIBear nonsense. Completely disagree.
There are tons of absolutely central results in mathematics that are understood by nobody: Classification of finite simple groups etc