Currently, in July 2026, the top forecasting bots are slightly below or equal to top humans. Extrapolating their progress, they should leave humans in the dust between 1 and 2 years from now. Unless the best humans are already near the theoretical maximum of how good even a superintelligence can be at forecasting. In which case AI forecasting is about to plateau.
This is a big deal because Sayash Kapoor and Arvind Narayanan (the "AI as Normal Technology" (AINT) folks) have set this as one of their lines in the sand. They claim that domains like chess and protein folding (and maybe math) are the exception and that in most domains, humans are already near the limit of how high performance can get. Specifically, they name forecasting and persuasion as domains with irreducible error. So if AI forecasting shoots past the top humans, that's a major blow to the AINT thesis. If not, it's a boost.
More background: Scott Alexander's post, "The AI Superforecasters Are Here". Key excerpt:
I generally disagree with Sayash and Arvind, but this is the prediction of theirs that I’ve thought about the longest, without being able to find any decisive refutation. It’s a great test case! If AI hits top-human level forecasting and then flattens off, maybe there’s something special about the human level, and S&A will also be right about superpersuasion, super-research, etc (at least for the near-term). If it keeps going, reaching heights far beyond the human maximum, then we should be concerned that it will do the same thing in other skills too. We’ll start to have a good idea which world we’re in within a year; after two years, the answer should be decisive.
To resolve this market, we'll use Scott Alexander's assessment. It's safe to assume he'll write about this again, since he's framing this as a key test of the AINT hypothesis. In any case, that's the spirit of the question: Will progress in AI forecasting falsify Kapoor and Narayanan's theory by July 2, 2027, one year after Scott's post? If so, AINT is wrong. If not, well, maybe AI just hit some speedbumps, but forecasting bots aren't crushing top humans by June 2028, that should be a meaningful update in the AINT folks' direction.
1,000also kind of hilarious in Scott Alexander's blog post that he appears so much more confident about the AI's prediction on respiratory infections at 7% than it's prediction of a US-China treaty to slow down AI at 1-2.2% 🤣 🤣
"I asked the AI superforecasters the probability of a US-China treaty to slow down AI, enforced by cryptographic verification of data center activity. FutureSearch said 1%; Preseen, 2.2%. This is unfortunate, because my movement has recently gone all in pouring its money and energy into making this happen. I admit I had an “uh oh” moment when I saw this number, but I haven’t given up or lost hope. I just figured it was outside the distribution of things that AI superforecasters are probably good at. This isn’t too crazy - in the past, “AI experts”, including many rationalists and safety advocates, have outperformed superforecasters at predicting the future course of AI."