I'll be referencing this
https://manifold.markets/calibration
Starting Brier score: 0.16781
To resolve as "better" or "worse" it should vary by more than 0.01
@Pazzaz Be the change you want to see in the market. I think the reason is mostly that this fluctuates a lot. I just checked and saw 0.175.
@Shump I did buy some "The same" before making the comment, so who knows, maybe I only made this comment to pump up the price! Maybe I faked the brier score to make it seem more likely, and no one would know! [insert evil laugh here]
Aren't Brier score and calibration very different things? Brier score is about how close you get to the true answer, calibration is about whether if you say something is x% likely, it happens x% of the time. You can be perfectly calibrated by always predicting 50% (or whatever the base rate of YES resolutions is) and still have a terrible Brier score. Brier score seems like it's largely going to depend on whether people make a lot of markets about near-certain events.
@StevenK In general I don't understand why forecasting people emphasize calibration to the extent that they do. Miscalibration is one source of error, but it doesn't seem like the most important one.