I keep seeing people talk about prediction markets as though they're some magic way to gain knowledge that isn't publically available. (e.g. #9 here.) As I explained in a response to that post, this seems overly optimistic to me. Markets are very good about updating on public information once it's known, but can they reliably do so beforehand?
See for example all the election markets that just track the polls until vote counts start coming in and then update based on tweets. What prediction value are those providing exactly?
Prediction markets are great at aggregating public information, but I'm not convinced they're actually that useful for prediction of future events, especially when those events are highly contingent upon a small group of people or tightly-guarded information.
This market resolves to YES if someone gives me an example of a market (doesn't have to be on Manifold) about some notable real-world event that had a mysteriously high or low price for quite a while, followed by the reason for that price later becoming known.
For example, if this market had been at upwards of 20% before any of the news broke anywhere else, that would convince me that prediction markets might actually work for fraud detection.
Otherwise it resolves to NO at the beginning of 2025.
I believe the market creator isn't active any more so we might need one of the mods to take a call on this.
This clearly resolves YES from the market below.
Irrelevant because Hyperion on discord said he provided insider information to dmayhem and mkualqueira Two arguments against: maybe the big YES holders didn't, themselves, have inside information. They were just making what they thought were positive EV based on inferences from other peoples' behavior. But the description doesn't mention insider info - the title is "eliciting knowledge", and the description says "that had a mysteriously high or low price for quite a while, followed by the reason for that price later becoming known". Mysteriously high price was satisfied - most manifold users were confused by the high price.
Maybe this event isn't "important" or "notable", it's just a few sentences, and open letters are released all the time. Notability is trivially satisfied - the statement was signed by executives of most big AI labs - DeepMind, Anthropic, OpenAI. And it's not just a restatement of something obvious like many open letters, AI companies openly acknowledging the risk of human extinction is quite dramatic and, to many, unexpected. This should satisfy 'importance'.
The description also mentions "especially when those events are highly contingent upon a small group of people or tightly-guarded information" - and that does describe this letter's release date.
@jacksonpolack It wasn't that tightly guarded, apparently it was at one point posted on Twitter and then deleted.
tightly guarded is not part of the resolution criteria. it's under the motivation for posting the question, labeled with an 'especially when'.
@Stralor this is the market in question. The market had a suspiciously high YES price for a while due to a few YES holders, while everyone (see comments) was very confused about this for a long time. Looks like the yes-holders may have been insiders or supergeniuses, because the market seems on its way to resolving YES. I think it satisfies all other criteria for this market to resolve as yes, the only possibly debatable one is whether it's a "major event". I'd say it definitely counts as a major event, because it has the world's top computer scientists and AI company heads putting out a joint statement.
https://manifold.markets/quinesweeper/will-there-be-another-wellrecognize?r=Z2FsYWdh
@galaga "Major" isn't mentioned; the title says "important" and the description says "notable", both of which seem reasonable to apply here.
@galaga First letter was not really important (not real impact) but somewhat notable. At least it had a stated objective. This statement is meaningless and harmless. It will have zero impact except being in newspapers.
The example given in the market is the FTX collapse, that's an important event. People writing letters, can we even call that an event ?
@Zardoru Do you believe that the world's AI leaders coming together and putting out a statement saying that the world needs to co-ordinate to mitigate human extinction due to AI will have no impact on the course of humanity in the long run? When I say "world AI leaders" I don't mean a couple cogsci professors and popular science writers. I mean more on the order of the CEOs of the world's leading AI companies (and potentially the most powerful people on earth in a decade) and the literal creator of backpropogation. Would you not say that a warning of human extinction by these people potentially impacts governments' AI policies and regulation and has significant effects on research in the field?
I am in general cynical of the effects of such letters, but this one seems quite on the nose. It reads, "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
@galaga it may not have an immediate impact, in the way that FTX did. But I think in the long run its second order and general equilibrium impact on regulation and policy compared to the counterfactual has a non-insignificant chance of exceeding that of the FTX collapse.
This is speculative, certainly it could turn out that tomorrow nobody reports on it and everyone has forgotten about it by the next week. But it could also incite a movement that's responsible for half of all AI regulation in 2030. It's hard to tell this early; history is a bit strange like that.
But I think it definitely should count for this market: I'd call such a statement by the world's AI leaders that is so far outside the public Overton window as "notable".
@galaga The new statement is not even on the first page of Hacker News.
Well, it's not even on the main page of "Center for AI safety" site http://safe.ai/ that means how they believe in it.
It has no impact because it is just a letter signed by a bunch of expert (most of them not really expert), and it's the second one in 3 months. It has no impact because it's only one sentence, there is no actual policy proposal, the wording is dull and we learn nothing.
We are inside the echo chamber of EA, Lesswrong, ACX... That why it seems important here. That's also probably why some Manifold participants had insider information. Outside the echo chamber people don't care much. And maybe they are right not to care.
@Zardoru I am sympathetic to this view. But note that the story is now on the first page of hacker news. It's also on many mainstream news outlets. Perhaps its main goal was to make the fact that AI leaders are in fact worried about xrisk by AI common knowledge, moving the Overton window and allowing people to express these ideas without being dismissed as extremist.
If you want to make a market about its impact, I would participate! It might be difficult to operationalize though.
@Zardoru 'AI experts say AI might cause human extinction' was the first item on the BBC news channel morning phone-in here in the UK this morning. I assume they were reporting on the letter.
Seems like this one has a shot?
https://manifold.markets/quinesweeper/will-there-be-another-wellrecognize
[this comment was previously temporarily deleted]
@Conflux https://manifold.markets/quinesweeper/will-there-be-another-wellrecognize what was wrong with linking this one here?
@Conflux Such "another well-recognized letter/statement on AI risk" would really be an important event ? Even the previous one was not really important. I was discussed a lot in some circles, made the news ok, but had not much impact. Another open letter about two months later, that would just demonstrate no one care.
Sorry for deleting my comment! I did so based on a suggestion from @firstuserhere, because I wanted to support their wish not to draw further attention to the AI letter market. Although, a Streisand effect later, it’s not like that helped much, and they pivoted towards a more confusion-oriented strategy :)
I don't expect to see prediction markets signalling still-private information, I do eventually expect to see prediction markets elicit private information by paying insiders to bet and then disclose the private information to move the market so that they can cash out immediately instead of waiting for the resolution date.
But I'd only expect to see this with real money prediction markets, and only if informants have enough going on as investors to care about cashing out sooner, and only in cases where the opportunity cost of waiting for closing date is lower than the value of keeping/not proving their secret, which, if they were keeping it secret in the first place, was probably substantial.
It... doesn't really work like that.
What makes prediction markets useful is that they tell you how likely the crowd thinks something is at a glance. You don't have to personally research every fucking question and follow every biased pundit, just look at one Pretty Good Guess and decisionmake off that.
Suppose someone buys shares up to an obscene price. People can speculate "insiders! insiders!" but it could also just be someone unusually confident despite having the same information as everyone else.
"Is it insiders?" will be discussed endlessly in comments and discords, but that's internal - it's not going to make it to the New York Times or go viral on twitter where people will see and speculate about it. Expert forecasters like Nate Silver will notice and try to figure out what happened, but their guess is good as any.
Then the price regresses back to approximately what it was originally - maybe a couple cents higher - cause no one has any new information.
It does happen that frequent market users will do original research and learn original facts and build their own models and it helps them win consistently. Which is kind of in the spirit of your question, though they typically don't share unless someone first goes "hey you've got a good head" and hires them.
People have an incentive to reveal in the comments any original research they do that will cause an early resolution of a market
@DylanSlagh I tend to reveal original research to see if the market moves (IE, am I good enough to convince people). That's a more direct ego payoff than accumulation of M in a few weeks
This would generally require scaling up prediction markets. For example in my whistleblower markets on Forbes Under 30 people, an insider could rally the price via insider trading. But almost nobody has heard of these yet. Even if anyone in that set is committing fraud, it likely wouldn't show up yet -- probably none of the potential whistleblowers are browsing Manifold.
The stock market seems like evidence against your position, assuming the scale is large. Nonpublic information can often affect stock prices.
I completely agree with your assessment, the value is in information aggregation, and prediction markets are unreliable mechanisms for discovering a high alpha in most applications, in the sense of, "novel predictions."
That being said, making novel scientific predictions is very difficult as it is a combination of mathematics and expertise. Having a high volume market with a lot of emotional back and fourth creates a social environment, with people trying to one-up each other with information for egotistical reasons.
So in that sense, individuals participating in a market, if they read and pay attention to the details of a market, can gain a high level of expertise vs. just reading the news or just following plain social media. Arguably as a research tool, it could allow individual researchers to move faster in either falsifying something or proving something.
But yeah, for the most part it's mostly a betting market, "prediction," is a misnomer as I have had on my profile for quite some time.
Here's where I talk about what predictions are, in the sense of novel predictions: https://www.youtube.com/watch?v=RNZNh4QVLfI
Complicating factors to this: More writings on the limitations and strengths of prediction/betting markets:
https://www.sciencedirect.com/science/article/abs/pii/S0169207018300657
https://www.sciencedirect.com/science/article/abs/pii/S0167923615001037
https://www.pnas.org/doi/abs/10.1073/pnas.1516179112
So basically, one could interpret going through past studies that were accepted as, "more on the true side," and then demonstrating how they were not replicable could arguably be binned as, "new information."
But then again, this market is for, "real world events:"
especially when those events are highly contingent upon a small group of people or tightly-guarded information.
https://manifold.markets/IsaacKing/is-bings-chatbot-gpt4
just wondering do you think your market on bing reflected general public opinion? Or is it notable enough?
The assumption here is that an insider/whistleblower would move the market. Instead they might just be relatively large positions. It looks like a lucky gamble. To spot this, one would have to compare it with the usual trading behavior. Much harder to prove.
I don't follow how this is supposed to demonstrate anything? It seems like the markets were at 70% before yesterday, which is the same as Yglesias's prediction, and then reacted after the news was announced. Am I missing something?
I'm uncertain whether to count the Doctor Who market. I was hoping for something a little more clear.
@IsaacKing I definitely wasn't expecting it to become the main topic of consideration.
It didn't feel to me like anyone else considered it to elicit knowledge about the show, since as I said below, very few people believed me no matter how I bet or explained myself. But I guess the point is that it was mysteriously high for a bit, and then the reason became known?
It's also worth considering that I was working entirely with publicly available knowledge. I think it should have resolved YES if I had inside knowledge of script details etc., but I never did. I did correctly sort out which insider leaks were real, though.
@IsaacKing I guess I wouldn't have thought of it as "important", though the description says "notable", and it does seem notable.
