Will Manifold's calibration be better at the end of 2023?
Basic
13
Ṁ1254
resolved Jan 1
100%71%
The same
9%
Better
20%
Worse

I'll be referencing this

https://manifold.markets/calibration

Starting Brier score: 0.16781

To resolve as "better" or "worse" it should vary by more than 0.01

Get
Ṁ1,000
and
S3.00
Sort by:

So it started as 0.16781, and is right now at 0.17295. This is a difference of 0.00514 which is less than 0.01. So the question would resolve to "The same" if it resolved now, correct? If so, why is "The same" only at 16% right now, when we're so close to the end of the year?

@Pazzaz Be the change you want to see in the market. I think the reason is mostly that this fluctuates a lot. I just checked and saw 0.175.

@Pazzaz I misread the decimal place

@Shump I did buy some "The same" before making the comment, so who knows, maybe I only made this comment to pump up the price! Maybe I faked the brier score to make it seem more likely, and no one would know! [insert evil laugh here]

Aren't Brier score and calibration very different things? Brier score is about how close you get to the true answer, calibration is about whether if you say something is x% likely, it happens x% of the time. You can be perfectly calibrated by always predicting 50% (or whatever the base rate of YES resolutions is) and still have a terrible Brier score. Brier score seems like it's largely going to depend on whether people make a lot of markets about near-certain events.

@StevenK In general I don't understand why forecasting people emphasize calibration to the extent that they do. Miscalibration is one source of error, but it doesn't seem like the most important one.

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules