Will Manifold get a better Brier score than the median answers of Blind Mode participants in the ACX 2023 prediction contest?
Will Manifold get a better Brier score than the median answers of Blind Mode participants in the ACX 2023 prediction contest?
28
550Ṁ2162
resolved Jan 4
Resolved
NO

Manifold has a group for questions from the 2023 ACX contest. Scott has released the answers of blind mode participants in this post. Once all markets in the linked group have resolved, this market will resolve according to whether Manifold's probabilities on these events as of January 10th 2023 12:01am have a lower (better) Brier score than the median answers from the data in the linked post.

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ780
2Ṁ428


Sort by:
predictedNO

Final Brier scores, (ignoring the N/A resolution) were:

0.172 for the blind mode mean

0.161 for the blind mode median

0.167 for Manifold

predictedYES 1y

@BoltonBailey I notice that I am confused

predictedNO 1y

@wadimiusz What are you confused about?

predictedYES 1y

@BoltonBailey Intutively, I'd think that markets are a better way of aggregating community's knowledge than simply averaging out every participant's opinion, and that switching mean to median wouldn't make that much difference. I observe the opposite and I don't really know why.

predictedYES 1y

Are the contest results posted somewhere?

predictedNO 1y

@SG Well, all of the markets have been resolved, so I based my calculations on that.

predictedYES 1y

@BoltonBailey Want to post them here? What were the final Brier scores?

predictedNO 1y

@BoltonBailey (There is one market that was N/Aed which there was some discussion about, but the resolution of that market doesn't actually affect the standings.)

predictedNO 1y

@SG Final brier scores are in the spreadsheet below, anyone feel free to check my work, I can try to re-resolve if I screwed up (I realize that I basically profited massively off of this, but it's not like the link down there hasn't been publicly showing Manifold losing for the past couple months).

predictedYES 2y

I tried to use the blind mode predictions when making my full mode forecasts, and my impression was they were just really bad, whether I looked at median, average, tried to exclude outliers, etc.

predictedYES 1y

@jack same.

predictedNO 2y

Google sheet for prediction help

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.
© Manifold Markets, Inc.TermsPrivacy