Once upon a time, Groups had a group chat, and in the chat for this group we sent a welcome message to people. Now this group serves no purpose...

Welcome

Join our meetup Friday 9th December 2022 at Easy Tiger from 6pm.

https://www.meetup.com/prediction-market-meetup/events/289741954/

About post for the group

Manifold Markets Austin Meetup

dynomight.net/prediction-market-causation

We all want to make good decisions. But it’s hard because we aren’t sure what’s going to happen.

Like, say you want to know if CO₂ emissions will go up in 10 years. One of our best ideas is to have people bet. For example, I might wager my $4 that emissions will go up against your $1 that they will go down. If lots of people bet in a market, the final prices combine everyone’s wisdom and information. If bets were to stabilize at a 4:1 ratio, that suggests that the odds of emissions going up are 4 times larger than the odds of emissions going down, i.e.

, you often need more information. Say someone comes up with a treaty to reduce CO₂ emissions. If you want to know if the treaty will work, you can have people make 

 bets. For example, I might bet my $9 against your $1 with these rules:

If lots of people bet in a market, and prices settle at a 9:1 ratio, that suggests that emissions go up in 90% of the worlds where the treaty gets signed, i.e.

You can run a second market to get the odds of emissions going up, conditional on there 

So here’s our question: Say you do that and the market thinks that emissions are more likely to go up with the treaty than without. That is, say 

. Does that mean that the market thinks that signing the treaty will cause emissions to go up?

People worry about prediction markets for lots of reasons. Maybe someone will manipulate prices for political reasons. Maybe fees will distort prices. Maybe you’ll go Dr. Evil and bet that emissions will go up and then go emit a gazillion tons of CO₂ to ensure that you win. Valid concerns, but let’s ignore them and assume markets output “true” probabilities.

Now, what would explain the odds of emissions going up being higher with the treaty than without? The obvious explanation is that the market thinks the treaty will cause emissions to go up:

Totally plausible. But maybe the market thinks something else. Maybe the treaty does nothing but voters 

 it does something, so emissions going up would cause the treaty to be signed:

In this chain of events, the treaty acts as a kind of “emissions have gone up” award. Even though signing the treaty has no effect on emissions, the fact that it became law increases the odds that emissions have increased. You could still get the same probabilities as in a world where the treaty 

 (albeit with internet points instead of money)

    P[launch | don’t declare] = 5.4%.

 but you can find an implied price using (1) the market for 

 and (3) the ᴘᴏᴡᴇʀ ᴏꜰ ᴍᴀᴛʜ.

. The obvious way of looking at this would be that NATO declaring a no-fly zone would increase the odds of a nuclear launch:

NATO and Russian planes clash over Ukraine

That’s probably the right interpretation. But not 

. For example, do we really know the mettle of NATO leaders? It could be that declaring a no-fly zone has no direct impact on the odds of a launch, but the fact that NATO declares one reveals that NATO leaders have aggressive temperaments and are thus more likely to take 

 aggressive actions (note the first arrow points up):

This could also explain the current probabilities.

Is this starting to sound familiar? If we had a market, I’m confident investors would assign a vastly higher price for 

Is that because reading dynomight makes you more charming and good-looking? Sadly, no—it’s because such people are more likely to value prime-grade autodidactism.

Does this remind you of another issue in probability and statistics? Like, say, the 

 using prediction markets to predict if banned commenters should be reinstated. The idea is that if you get banned, you can write an argument that you will be good and then investors would bet on the odds that Scott would reinstate you if he read your justification, i.e.

This might not work. Suppose that Scott currently pays an assistant to pre-screen the justifications. If the assistant thinks there’s at least an 80% chance someone will be reinstated, they forward their justification to be read, while the rest are ignored. (The assistant gives 

 “ideal” predictions, in the sense that it’s impossible to be more accurate given the same information.)

This distorts the market. Assuming people know about the assistant, the market will give at least an 80% probability of reinstatement to 

, regardless of how bad it is. (You are guaranteed to make money over time by blindly taking any bet with odds lower than 80%. So people will always bid the odds up to 80% or higher.)

What went wrong? The problem is that the quality of the justification influences if it gets read:

             ↓          ↘  

This same distortion of probabilities happens with almost any selection based on the quality of the justifications, though I can’t figure out how to make the general case without some grungy calculations.

But here’s one example: Suppose that Scott looks at only those cases where the prediction market is above 25%. This will also distort the probabilities upward. Suppose that the market is currently at 30% but you think the true probability is 20%. Consider two cases:

That doesn’t seem fair, does it? In the situation where you’re right, the bet is likely to be canceled. So you’ll notice this and not bet. When people who think the probabilities are low drop out of the market, the probabilities get distorted upwards.

The solution to this is to read justifications chosen 

, without any consideration of the quality of the justification or market prices. This breaks the link from the justification to the event of being read:

             ↓             

Look at that: To make things work, all we needed to do was promise to our conditional prediction market that we will do the conditioning 

Where are we? Let’s review. Here’s the typical 

So far, this article has made this argument:

Basically: If you run a prediction market to predict correlations, you get correlations. If you run a prediction market to predict the outcome of a randomized trial, you get causality. But to incentivize people to predict the outcomes of a randomized trial you have to actually 

So that’s the problem. What can we do about it? There are several options.

One option is to think carefully about the system you’re studying. If the causal structure is OK (no reverse causality, no confounders), then you’re OK, and a conditional prediction market will give you what you want. But do be careful, because if you’re wrong you’ll get meaningless results.

There’s an interesting wrinkle here which is that I guess all that you 

 need is that people betting in the market 

 the causal structure is OK. It seems dangerous to try to exploit this, though.

We solved the problem with banned commenters by committing to choosing actions randomly. This option always works. But there’s a cruel irony: The whole point of prediction markets is to make good decisions. If you have to behave 

 to get good predictions, then what are we even doing here?

Fortunately, there’s a good (and well-known) alternative, which is to randomize decisions 

. You tell people: “I will roll a 20-sided die. If it comes up 1-19, everyone gets their money back and I do what I want. If it comes up 20, the bets activate and I decide what to do using a coinflip.”

What’s nice about this is that you can do whatever you want when 1-19 come up, including making your decisions using the market prices. But you must make decisions randomly 

, because if bets never activate, no one will waste their time betting in your market.

This is elegant. Oh, and by the way are you THE NSF or DARPA or THE NIH or A BILLIONAIRE WHO WANTS TO SPEND LOTS OF MONEY AND BRAG ABOUT HOW YOU ADVANCED THE STATE OF HUMAN KNOWLEDGE MORE THAN ALL THOSE OTHER LAME BILLIONAIRES WHO WOULDN’T KNOW A HIGH ROI IF IT HIT THEM IN THE FACE? Well how about this:

Still, I really hope that NATO doesn’t randomize the decision about a no-fly zone.

Consider again the case where people could bet on the odds of a commenter being reinstated. It would be nice to use a market to do filtering, i.e. only read the justifications that have a probability above 25%. But we saw that would distort the market.

There’s an alternative betting structure that could mostly solve this problem. The idea is that if the final market probability is above a threshold you always read the justification. If the final market probability is below a threshold, you only read the justification a fraction of the time, but you make the bets correspondingly larger. For example, we might agree to these rules:

Regardless of if the market is above or below 25%, 

, I will win $1 if I’m right that the person will be reinstated and you will win $4 if you’re right that they won’t. So this market will still incentivize people to bet their true beliefs.

Imagine a prediction market for the odds of a war conditional on who wins a presidential election. You couldn’t interpret this causally. For example, maybe candidate X is less likely to cause a war, but if it already looks like war is looming, then people might vote for candidate Y.

But we can’t randomly pick presidents. (That would interfere with our elections which are 

 just as bad as choosing a winner at random.)

 a market that only activates if the votes are very close to even, so that the outcome is 

This isn’t perfect, because an election being close might influence other variables. For example, imagine that candidate Y likes to undermine the legitimacy of elections. If there’s a clear result it won’t matter, but if X wins narrowly, riots would start and global adversaries might take advantage of the chaos to start wars. In a case like this, a “close call” prediction market might make it look like X is more likely to cause a war, even though X is better under most circumstances.

Still, this is a good idea—it’s not that far-fetched to just assume that things like that don’t happen. You can probably invent other markets with the same principle of “bet that activates if a natural experiment happens to occur”.

Robin Hanson has also suggested “fire the CEO” prediction markets where people bet on the future price of a company, conditional on a CEO being fired (or not). If such a market says the price is more likely to go down if the CEO is fired, that doesn’t necessarily mean the CEO is good because you might imagine something like this happening:

An obvious way to deal with this is to use time: Bet on the stock price 2 years from now, conditional on the CEO being fired in the next month.

This is still no guarantee because it only eliminates reverse causality, not hidden confounders. Maybe everyone knows the CEO is terrible, but hard to fire. But the company was just bought by some billionaire and there are weird rumors that the billionaire hates the CEO and wants to fire them and then drive the company into the ground out of spite:

If you learned that the CEO gets fired soon, you’d have to conclude that the billionaire really does hate them and plans to go on and destroy the company. So even though the CEO is terrible, you’ll have conditional probabilities that make it look like the CEO is good. Time is no magic bullet.

Controlled conditional prediction markets

Finally, one could add control variables to the prediction market. Say you want to know if you should fire the CEO, but you’re worried about things being confounded by a recession. Then you could run four prediction markets for the odds of the stock going up conditioned on each combination of the CEO being fired (or not) and there being a recession (or not). If the CEO truly is crap, you’d expect that

Unless you're careful, conditional prediction markets have all the same problems as observational studies.

Prediction market does not imply causation

FTX is hosting a competition asking individuals to help change their minds on AGI xrisk.

This group is not directly affiliated with the 

Here are some markets to predict on the results of the competition.

FTX Future Fund's AGI Worldview Prize

Manifold team's predictions about what will happen this year!

Manifold's 2023 Predictions

Click here to view the World Cup Tournament page.

The intent of this tournament is to reward individuals who are best at predicting and live trading about the FIFA 2022 Qatar World Cup.

A total prize pool of $500USD will be shared amongst the top 5 traders.

A couple of new markets will be added weekly in addition to the existing collection.

Trading will cease at half-time for markets predicting the result of a match. This is to allow some live trading while ensuring users who are predicting in advance aren’t significantly disadvantaged by people who are on the site 24/7.