I propose adding "Specifically, my credence on the proposition 'Using 4 months of individual-engineering time, a pre-2026 AI could be fine-tuned and scaffolded to out-perform, on mean brier score, over all binary questions on Metaculus".

If no one objects within a week, I'll add this.

"If this does not happen, and no negative result comes out in the last quarter of 2025, then this question resolves to my subjective credence that this could be done with an existing AI system and scaffolding."

Does this include finetuning?

@NoaNabeshima Yes my subjective credence includes limited fine-tuning things like the berkeley group's level of fine-tuning are fine.

I think it can be done in principle it's just not clear it will be done in practice

Unless anyone objects, I'll clarify the constraint that this out-performance should hold on average for at least 50% of the questions on Metaculus in a prospective study. Obviously if this ends up depending on my credence, I'll be taking into account other results e.g. the below.

@JacobPfau wdym by "hold on average for at least 50% of the questions"? if they outperform it'll be an average of theirs vs an average of metaculus, I would think?

@Bayesian You're right that the question phrasing implied all questions, though I didn't specify binary vs time series etc. I'll come back to this tomorrow.

Regretting the "I will not participate in this market" anyway https://arxiv.org/pdf/2402.18563.pdf

@JacobPfau FWIW I'm at 90% on this.

Excellent operationalization!