Will Matthew and Tamay win their bet on longer AI timelines by 2026?
Plus
16
Ṁ6042026
9%
chance
1D
1W
1M
ALL
From https://www.lesswrong.com/posts/X3p8mxE5dHYDZNxCm/a-concrete-bet-offer-to-those-with-short-ai-timelines
@MatthewBarnett and Tamay believe that AI timelines are longer than expected.
Specifically, they were offering "1:1 odds that no more than one of the following events will occur by 2026-01-01:
- A model/ensemble of models achieves >80% on all tasks in the MMLU benchmark
- A credible estimate reveals that an AI lab deployed EITHER >10^30 FLOPs OR hardware that would cost $1bn if purchased through competitive cloud computing vendors at the time on a training run to develop a single ML model (excluding autonomous driving efforts)
- A model/ensemble of models will achieve >90% on the MATH dataset using a no-calculator rule
- A model/ensemble of models achieves >80% top-1 strict accuracy on competition-level problems on the APPS benchmark
- A gold medal for the IMO Grand Challenge (conditional on it being clear that the questions were not in the training set)
- A robot that can, from beginning to end, reliably wash dishes, take them out of an ordinary dishwasher and stack them into a cabinet, without breaking any dishes, and at a comparable speed to humans (<120% the average time)
- Tesla’s full-self-driving capability makes fewer than one major mistake per 100,000 miles"
Resolves YES if they win their bet (no more than one of the above occurs by the end of 2025).
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Sort by:
I just wrote a retrospective on the bet. I think it's highly likely that we'll lose, unless something weird happens, like people stop publishing on all the benchmarks that we used for the bet. I think what happened is that (1) my AI timelines were simply too long, and (2) I wasn't thinking very carefully about the milestones, and didn't realize that extrapolating past trends reveals that we are expected to lose unless progress slows down by a lot.
Related questions
Related questions
Will AI pass the Longbets version of the Turing test by the end of 2029?
56% chance
Will an AI solve a Millennium problem by EOY 2027?
29% chance
Will Matthew and Tamay win their bet on longer AI timelines by 2030?
2% chance
Before 2030, will an AI complete the Turing Test in the Kurzweil/Kapor Longbet?
56% chance
Will Ray Kurzweil win his 2029 Turing Test bet with Mitchell Kapor?
56% chance
Will AI wipe out humanity before the year 2026?
1% chance
Will there be substantive issues with Safe AI’s claim to forecast better than the Metaculus crowd, found before 2025?
77% chance
Will any of the remaining Clay Millenium problems be solved with substantial help from an AI before 2030?
43% chance
Will AI pass the Winograd schema challenge by the end of 2024?
64% chance
Will Bryan Caplan win his bet with Matthew Barnett on whether an AI can pass his exams in 2029?
7% chance