I'm rerunning / updating my [Manifold calibration analysis code](https://github.com/vluzko/manifold-markets-python).

Considering all resolved binary markets, will Manifold be correctly calibrated halfway between the start and close of the market?

Procedure: All markets get bucketed into 10% intervals (5% at the edges). This gives us 11 buckets: 2.5%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 97.5%. Then we compute what fraction of each bucket actually resolved YES. Perfect calibration would be if 2.5% of the 2.5% bucket resolved YES, 10% of the 10% bucket, etc.

I will consider Manifold well-calibrated if 7 out of the 11 are within 5 percentage points of perfect calibration (so if 5% to 15% of 10% confidence markets resolve YES then the 10% bucket is well-calibrated, if 7 buckets are well-calibrated this market resolves YES). (This is not an ideal measure of calibration but I want something simple so the market resolution criteria is easily understandable).