@Bayesian '>14h' can resolve as 'yes' because of the Opus 4.6 measurement (14 hours and 30 minutes). Thanks!

The members of the AI futures project have given an update and they appear to now be relying on the 80% time horizon length graph from METR for their predictions rather than the 50% time horizon length graph (correction: they have always used the 80% time horizon length). This implies that a 50% time horizon is not enough. While I think markets for 50% time horizons are useful, I now think that more attention needs to be paid to 80% time horizon lengths.

@MaxLennartson Source: https://www.aifuturesmodel.com/#section-timehorizonandtheautomatedcodermilestone

@MaxLennartson Well, now the 50% time horizon measure is saturated.

@Haiku Sorry I am confused about your comment?

@MaxLennartson You had said that markets for the 50% time horizon were useful. But with the release of the 50% time horizon for Claude 4.6 Opus, METR said that it's hard to measure now because the benchmark is saturated. Basically, we don't actually know what the time horizon of Claude 4.6 Opus is. They're continuing to work on updating the time horizon benchmark, but the new version might be saturated at 50% by the time it gets released. I expect them to retire the 50% benchmark and add a 95% benchmark.

@bens

@Bayesian hmmmm

@Bayesian might depend on the methodology announced by METR for those longer tasks

@Bayesian but I’ll take some now just for kicks

I very roughly polled METR staff (using Fatebook) what the 50% time horizon will be by EOY 2026, conditional on METR reporting something analogous to today's time horizon metric.

I got the following results: 29% average probability that it will surpass 32 hours. 68% average probability that it will surpass 16 hours.

The first question got 10 respondents and the second question got 12. Around half of the respondents were technical researchers. I expect the sample to be close to representative, but maybe a bit more short-timelines than the rest of METR staff.

The average probability that the question doesn't resolve AMBIGUOUS is somewhere around 60%.

@Bayesian am i misinformed or outdated now? If the doubling period was 7 months or whatever these estimations seem quite optimistic on an increase in the doubling speed!

@No_uh the doubling times havent been 7months, closer to 4.5-5 months

@Bayesian were they never 7 months? am i just misremembering? or did they seem to shift not too long ago?

@No_uh oh yeah they were 7 months, and for a while they were consistent eith 4 month to 7month range, and that uncertainty is slightly narrowing over time

@Bayesian Yes, that makes sense. it looks like I just am not up to date on the narrowing. I'm only human lmao, seems I already cannot keep up. Welp, enjoy my free mana everyone ;)

exit: and thank you as always Bayesian for responding!

@Bayesian I am personally going with a five month doubling time.