This refers to stage 1 of the prediction contest ("blind mode") to see whether we performed as well as we did last year where prediction markets scored at the 99.5th percentile (beating 506 of 508 participants).
See here for the discussion of the 2022 results:
https://astralcodexten.substack.com/p/who-predicted-2022
Here is the group of markets for the questions: https://manifold.markets/group/acx-2023-prediction-contest
If the Astral Codex 2023 contest results writeup states Manifold performed better than last year's result (at or higher than 99.5th percentile compared to blind mode results), this will resolve yes.
Related, but regarding Full Mode, where contestants may use Manifold or other markets or research.
I'm assuming this will be restricted to comparison against 2023's "blind mode" participants, in order to be most similar to the comparison last year?
Betting on regression towards the mean - I think Manifold's "mean" is significantly higher than the overall mean, but 99.5 percentile is a high bar and I think some chance was involved.
@DanStoyell Same strategy here. 99.5% is an extremely high bar.
On top of that there's rounding issues this far out on the tails, e.g. if there are less than 200 participants, the requirement is to win the contest.
It wasn't clear from Scott's writeup: When did he take the Manifold predictions? Was it on the last day for other contest predictions, or was it later?
@NickAllen I'm not entirely sure, that's why the resolution criteria is based on what Scott says our performance is as presumably, he will keep the method of comparison the same between years. If it isn't clear in the writeup for this year's contest I'll send a message and ask him.
@DavidChee Sam says on lesswrong: These are supposed to be market prices as of February 14, 2022, which was the last day for contest submissions.
