How many participants does a manifold market need to become sufficiently predictive (80% probability of being right)?
44
257
3009
1-4
5-19
20-49
50-99
100+

Get Ṁ1,000 play money
Sort by:

Technically, a market is never "right," as more predictors could still make it more precise (unless it has reached 100% or 0% for something with a guaranteed outcome). However, I have a hard time otherwise defining "sufficiently predictive." My convoluted working definition is "at least 80% of markets with this number of participants are within 5% of the asymptotic value as the number of participants approaches infinity."

Depends on the market, but most stabilize around 5-10 traders in (especially if they're about niche topics that a small number of people have a very large interest in.)

80% probability of being right (if by right means that it is above 50% if it happens and below 50% if it does not happen) is a pretty low bar. Many things are so close to 100% and 0% that a single person could easily get above 80% with some selective betting.

@SophusCorry I mean an 80% chance that a market resolves with the implied probability. For example markets with a 70% chance to resolve as YES will have a 70% chance of resolving YES at least 80% of times.
Which makes me think that we should probably factor in time dependencies here, since predictions will be a lot more accurate right before the occurence of an event, but also a lot less useful.
A more precise question might be: "What is the minimum number of participants at which a median manifold market at its half time will resolve at least 80% of the time with the agreed upon chance"
Or something like that. I wrote myself in a corner here.

if you look at https://calibration.city/manifold the calibration is pretty good after 5 traders, even with a low number of trades, though at there is a bias where the numbers are closer to 50% than what they should be

@NihilCranes I guess that makes the poll obsolete

@Symmetry nah it's still a good question