holy legalese batman

Fair enough! Edited.

with model labels:

@0xseraphim

Market was designed such that only a significant acceleration on AI capabilities will lead to a YES settlement.

@MNX the uncertainty bands would be too high (same issue as METR time horizons >16h)

unless they pin Fable 5.5 to 170 or similar

since extrapolating from Sonnet 3.5 = 130 and GPT-5 = 150

is pinning at least an uncertainty of say 10 or 20% (i.e. if the “true” gap between Sonnet 3.5 and GPT-5 is likely between 16 and 24)

then the definition of 200 ECI varies from Dec 2027 (+2.4 years from GPT-5) to Feb 2029 (+3.6 years from GPT-5) at the ~Jan 2025 rate of progress.

I can try to quantify this but look at how AECI in the recent Anthropic system cards reports both local and global uncertainty. Local uncertainty should be on the order of 1 while global would be on the order of 5 to 10, meaning by the end of 2027 (where we expect ECI ~190) it’d be on the order of 10 to 20 barring some change in methodology

Yeah it looks like saturation makes the problem worse since if the relevant benchmark a year from now is something GPT 5 and Sonnet 3.5 both score 0 on (or worse if 3.5 is better than GPT 5 since something about Anthropic pretraining is useful for getting 1 problem right)

that data point only widens the uncertainty on GPT-7 / Fable 6.5

—-

Looks like I might be exaggerating

So good market I guess