The mean poll response before the end of January was 76% (this is the value this market resolves to). The median response was 95%. Here are charts showing the distribution of responses - 60% of responses were >=95%, 20% were between 50% and 95%, and 20% were <50%.

I think using the mean is definitely not the best aggregation of the poll responses, although it's not clear what the best one to use is.

@jack I think the fact that there was incentive to choose 0 if you were buying no is definitely why there were so many of that response. I wouldn't see the problem with using a median as it would probably give you better consensus, though it wouldn't represent peoples' actual extreme takes. (I'm using extreme to mean 1 or 10, not that the opinion is unreasonable)

@DesTiny Nobody answered 0, the lowest response was 2%, the histogram is showing several responses in the 0-20% bucket.

I did specify that you had to answer your honest beliefs. Of course, motivated reasoning is a thing, but I didn't see any clear indications of poll manipulation (except one person who didn't see the instruction to answer honestly, submitted several answers that explicitly said they were manipulation, and then submitted another answer with their actual belief after they saw the instruction - I removed their other answers from the analysis of course).

Median is one useful measure, but it doesn't reflect all the information we're interested in. There might be a way to average log odds or something like that to reflect that there is a big difference between 99% and 98% that isn't reflected in the mean.

@jack sorry yeah I forgot it was a bunch of questions, I'm not sure what the best means would be

@DesTiny There's only one question that was actually scored here. The other questions were to try to understand how well calibrated the poll answers were. Unfortunately, they weren't very helpful because nearly everyone said they weren't familiar with Lehman brothers

@jack Yeah, I think multiplying the odds together would be reasonable to aggregate a bunch of point estimates?

@jack This can resolve.

I bought "no" supposing that the average probability of the poll would be less than the 89% the market was at.

But actually the market will only resolve YES or NO, and the threshold for that is the poll being at probability 50%. So the actual price for buying here is not "what % do you believe the poll will come out as?" but rather "what's the probability that this poll will come up greater than 50%"?

These are very different quantities.

@AdriaGarrigaAlonso I presume the market would actually resolve to the average poll probability (markets can be resolved with a fractional result, so e.g. resolving to 0.93 means you get 0.93 for your yes shares and 0.07 for your no shares)

Yep, what @MartinModrak said

Weird way to do it. Say the poll reaches 99% - won't people who predict "no" get virtually guaranteed profits?

Seems like more of a game theory experiment than a poll.

@StanPinsent You might be mixing up the poll and the market. The poll results are hidden until it closes, so there's no way for anyone to see that the poll has reached 99%. If the market reached 99%, then it would be profitable to predict NO if you believe the poll will be less than 99% - there's no weird game theory there, it's just predicting as normal.

@jack ok that makes a lot of sense! Thanks

@StanPinsent No, see my comment above -- the probability that the market resolves YES is not the same as the average result of the poll

more like sam bankman freed for being totally innocent of all charges

@ForrestTaylor lol

I was about to start arbitraging with your two related markets and then realised that not every participant in your polls will be a perfect bayesian.

@Fion Not only that, but the polls will very likely be answered by different people, so there's a lot of random noise and/or selection bias that can occur.

However, I think it is probably true that the expected value of the markets should be reasonably close.