Will Manifold get a better Brier score than the median answers of Blind Mode participants in the ACX 2023 prediction contest?
28
244
550
resolved Jan 4
Resolved
NO

Manifold has a group for questions from the 2023 ACX contest. Scott has released the answers of blind mode participants in this post. Once all markets in the linked group have resolved, this market will resolve according to whether Manifold's probabilities on these events as of January 10th 2023 12:01am have a lower (better) Brier score than the median answers from the data in the linked post.

Get Ṁ200 play money

🏅 Top traders

#NameTotal profit
1Ṁ780
2Ṁ428
Sort by:
predicted NO

Final Brier scores, (ignoring the N/A resolution) were:

0.172 for the blind mode mean

0.161 for the blind mode median

0.167 for Manifold

predicted YES

@BoltonBailey I notice that I am confused

predicted NO

@wadimiusz What are you confused about?

predicted YES

@BoltonBailey Intutively, I'd think that markets are a better way of aggregating community's knowledge than simply averaging out every participant's opinion, and that switching mean to median wouldn't make that much difference. I observe the opposite and I don't really know why.

predicted YES

Are the contest results posted somewhere?

predicted NO

@SG Well, all of the markets have been resolved, so I based my calculations on that.

predicted YES

@BoltonBailey Want to post them here? What were the final Brier scores?

predicted NO

@BoltonBailey (There is one market that was N/Aed which there was some discussion about, but the resolution of that market doesn't actually affect the standings.)

predicted NO

@SG Final brier scores are in the spreadsheet below, anyone feel free to check my work, I can try to re-resolve if I screwed up (I realize that I basically profited massively off of this, but it's not like the link down there hasn't been publicly showing Manifold losing for the past couple months).

predicted YES

I tried to use the blind mode predictions when making my full mode forecasts, and my impression was they were just really bad, whether I looked at median, average, tried to exclude outliers, etc.

predicted YES

@jack same.

predicted NO

Google sheet for prediction help

More related questions