Will pre-2026 AI out-forecast the Metaculus community?
30
2.4k
2026
45%
chance

Will an AI system out-perform the Metaculus community prediction before 2026? Any amount of scaffolding is allowed.

If this does not happen, and no negative result comes out in the last quarter of 2025, then this question resolves to my subjective credence that this could be done with an existing AI system and scaffolding.

I will not participate in this market.

Get Ṁ600 play money
Sort by:

I propose adding "Specifically, my credence on the proposition 'Using 4 months of individual-engineering time, a pre-2026 AI could be fine-tuned and scaffolded to out-perform, on mean brier score, over all binary questions on Metaculus".

If no one objects within a week, I'll add this.

bought Ṁ100 YES

"If this does not happen, and no negative result comes out in the last quarter of 2025, then this question resolves to my subjective credence that this could be done with an existing AI system and scaffolding."

Does this include finetuning?

@NoaNabeshima Yes my subjective credence includes limited fine-tuning things like the berkeley group's level of fine-tuning are fine.

bought Ṁ150 NO

I think it can be done in principle it's just not clear it will be done in practice

Unless anyone objects, I'll clarify the constraint that this out-performance should hold on average for at least 50% of the questions on Metaculus in a prospective study. Obviously if this ends up depending on my credence, I'll be taking into account other results e.g. the below.

@JacobPfau wdym by "hold on average for at least 50% of the questions"? if they outperform it'll be an average of theirs vs an average of metaculus, I would think?

@Bayesian You're right that the question phrasing implied all questions, though I didn't specify binary vs time series etc. I'll come back to this tomorrow.

Regretting the "I will not participate in this market" anyway https://arxiv.org/pdf/2402.18563.pdf

bought Ṁ10 NO from 47% to 46%

@JacobPfau FWIW I'm at 90% on this.

Excellent operationalization!

More related questions