Do they count as a surprise guest if The Chicks were announced ahead of time? https://www.npr.org/2024/08/22/g-s1-18907/the-chicks-national-anthem-democratic-national-convention

Once again, in questions I'm curious to know the answer to, Prediction markets utterly fail.

I fully expect the Dwarkesh "Biggest guest ever" to not be predicted correctly.

I know, I know, all the studies of calibration by midpoint of market... My personal anecdotal experience has been that they are close to useless.

Has anyone else felt that way?

Can you remind me of some times when they worked well?

https://manifold.markets/calibration

Did you read my second paragraph??

"I know, I know, all the studies of calibration by midpoint of market... My personal anecdotal experience has been that they are close to useless."

I have a hunch that a lot of that comes from sports or similarly easy to predict questions.

regardless, this is personal experience not a statistical analysis like I said very clearly.

Biden dropping out was a big one for me. The most important question of the year imo, and prediction markets (especially manifold but not only) did not help at all

I felt prediction markets were very far ahead of conventional wisdom on Biden dropping out. Maybe you had a different reading of conventional wisdom. If you knew all along but chose not to bet on that to profit, I tip my hat to you, very noble.

No, you misunderstand my complaint.

I didn't know all along, and I didn't know the answer to this all along. So I look to prediction markets to tell me. Here's the point: They aren't helping me!

This is not about me being right, I didn't bet on this at all. This is about me downweighting how much to use Markets in my daily information gathering

To me personally, I feel like the real value of prediction markets is keeping a track record of who's good at predicting, not necessarily being the predictions themselves. Their purpose is to separate the Alan Lichtman's from the Nate Silver's, then in the future you pay attention to Nate Silver instead of the market.

Yes that's why I started with "conventional wisdom". The future is unknowable. All that matters is relative comparisons. Prediction markets were comparatively accurate about Biden dropping out than the other sources in my media diet. So it was a straightforward win for prediction markets there. But if you had sources that told you Biden would drop out earlier than Polymarket did, then it isn't a win for prediction markets, and you must have had some better sources than I did, fair enough.

However, there needs to be some alternative source that is more reliable for prediction markets to lack their utility. The fact that no one can tell you who will be president in 2025 or who will win the next super bowl does not make any prognostications about those events useless—that's just a trivial statement that some uncertainty is irreducible. (Duh?).

Markets like this (long list of multiple independent answers) have extraordinarily low liquidity. You can see relatively small single trades shifting probabilities by orders of magnitude. And the market only ran for a day. So there aren't going to be particularly diligent trades and I'd expect it to be relatively poorly calibrated.

It's also not really that consequential of a question. Maybe a better way to evaluate prediction markets: Is there anywhere else on the public Internet that would have given you better information on the question?

@mongo

That's a fair point, but it's a lot more work. maybe I'll make a CLI which returns SemioticRivalry's thoughts on current events or something - not a bad idea.

@Ziddletwix

A big chunk o my daily news is Axios newsletters, they were talking (pretty seriously) about Biden dropping out within 3 days after the debate. (for the record, I was wrong too - bigly, I doubted them a bunch) you cannot claim prediction markets had a "Straightforward win".

@cthor

It's more of a straw that broke the camels back. no this is not important in any sense, but I was very curious. I am also very curious about who'll be Dwarkesh's guest, and I notice myself looking at the odds and saying to myself; Eh, I don't trust this at all anymore.

To clarify, I'm complaining as someone who wants to use this site as part of my information diet etc. And I'm finding it less useful than reading semafor etc.

(also to make manna, obv....)

I think the big advantage of prediction markets is that if you can find a better source for your information diet, you can bring the knowledge you gain to Manifold and make a bunch of mana yourself correcting Manifold's numbers

@mongo

Ya, but making manna is not a big life goal of mine! knowing the truth about reality is

I ask for one case where reading the odds here (or polymarket etc.) gave you something more than your friends who don't.

Something comparable to subscribing to Politico PRO or Semianalysis or Stratechery or something equivalent...

What would be the market that after you looked at it you knew more than random dude who follows the news and has opinions?

A big chunk o my daily news is Axios newsletters, they were talking (pretty seriously) about Biden dropping out within 3 days after the debate. you cannot claim prediction markets had a "Straightforward win".

This is hard to follow. Polymarket was at 20% for Biden dropping out before the debate. Do you have a link to an Axios article from before the debate that you think gives such a clear prediction that he might seriously drop out? Prediction markets made me take that possibility seriously, without prediction markets I would not have taken it seriously. That's my simple TL;DR example for a concrete example where prediction markets informed me beyond what reading those sources would do.

Within hours of the debate, those Polymarket odds sharply increased to the 35-45% range (that's quite likely!). Here's Axios' debate summary article from the next day—there is no serious mention that Biden might drop out of the race. If Axios had included a link to Polymarket and added "by the way, bettors now think Biden has a 40% chance to drop out of the race", the article would have read quite differently (and in this case, it would have been more predictive).

Yes, by 3 days after the debate, Axios & prediction markets & my friends & my mom all took Biden dropping out quite seriously.

The main market I find useful for sanity checking me is just Kamala vs Trump election odds. Otherwise I'm not really interested in specifc events, I like big picture predictions like which economic policies are better, which prediction markets are iffy for anyway since there are so many confounding factors.

I am more into getting mana, it's a really fun game to me, so I'm usually on the lookout for the least accurate markets too so I can make my own mana

@mongo @Ziddletwix @cthor

I finished the CLI (phew!), here it is:

https://github.com/luvchurchill/superforecasters-only

hopefully instructions are clear enough, feel free to open a PR....

Not Beyonce https://x.com/thr/status/1826805194707104170?s=46&t=nwWh6Z5UIzLB_3nzFDJjXQ

If a special guest was supposed to appear but last minute was unavailable to make it, like because of a traffic accident or health emergency, would they resolve yes or no?

Must be shown live. No.

https://www.rollingstone.com/politics/politics-news/dnc-beyonce-prepping-arrival-has-not-confirmed-1235085940/

@MichaelWheatley @ManifoldPolitics Can you please also add to the DNC Last Day dash? Thank you!

We've already got the other one and the two together would be redundant, IMO

Great, thanks.

Wow we created our markets nearly simultaneously

Does showing up in a video montage of celebrities singing count?

No.