🏅 Top traders

Manifold's calibration score hasn't improved using this methodology...but it has if you filter out nonpredictive markets, which you can do from @wasabipesto's calibration site!

My take: the probability of this happening is less than 5%.

Why I believe this: The vast majority of times when I check the calibration page, the Brier score is more than 0.17. Today I did this more systematically, and all 5 out of 5 samples had Brier higher than 0.17 (in fact, higher than 0.171). I'd estimate the frequency of <0.17 samples is maybe 0.1, and would be quite surprised if it was more than 0.2. Even if the frequency was 0.2, the probability of the market resolving YES would be just 3.3%.

(Why I share this now: I'm already heavily invested in the market and am running out of liquidity - I wouldn't mind if the market started to agree with me and I could take some of my mana out.)

Seeing below 0.17 again :)

Do we have an easy method of drawing ten samples? If not, can you do an interim one?

It was showing me 0.169 the other day and that seemed significant enough for me to update a bit towards it being possible.

It looks like there's pretty good alpha for a bot that just buys a smidgen of NO in every market.

I feel like most* markets get calibrated to within about 10% of reality within single-digit numbers of betters. Once it’s roughly calibrated, there’s less profit to be made usually(but not always)

@JohnSmithb9be I’m not actually sure I can explain why I feel this way so if anyone who’s a stats expert can critique this opinion I would welcome it!

Reasons why calibration might improve:

• We grow and have more users trading in markets.

• We continue to unlist whalebait and non-predictive markets (which tend to be less calibrated).

• More people create bots and bot trading algorithms get better (partially a function of having more training data)

@SG Increasing users might actually decrease reliability. Probably some growth is good, but growth that is too fast I suspect will lure in what we call "dumb money" in betting market circles.

@AaronKreider Dumb money just means more profit for the rest of us 😋