, who we commissioned to write an unbiased, outsider view on the Whales vs Minnows drama that recently consumed Manifold. Enjoy! -David

 where you borrow a bunch of stocks, sell them, hope the price goes down, buy them back at a lower price, return the stocks, and keep the difference. Fees and interest aside, it’s a pure price bet - you set a neutral win/lose level when you buy the stock and hope it stays under the bar. 

Short selling is a weird enough process that there 

. There are usually enough market-enhancing positive effects to it that the government never actually gets around to banning it. But make no mistake: there are 100 ways to skin the “get yourself in financial or legal trouble with shorts” cat. Shorts are a bet made on loan, and it’s trivial to get yourself in so deep you’d have trouble fishing yourself out.

Notably, if you are a big fish that makes itself vulnerable by getting deep into the short pond and enough small players notice, 

The reason I’ve been thinking about short selling this week is that I’ve been trying, as a layman, to find the closest normie-world equivalent to what happened to the Isaac King Whales/Minnows market controversy. And I can’t. Because in this case a weird pretend-money prediction market and too much dedication to winning at all costs came together to create something entirely new.

There’s no name for what happened, although I am going to pitch “Kingbreaking” as a possible choice. I’ll do my best to explain what went on from an outsider’s perspective and how Manifold’s biggest whale managed to lose a massive amount of Mana and $30,000 real-world dollars in a market he himself created.

entry about whether all the spiders on Earth (which are nearby) cumulatively exert more gravity on you than the much-farther-away sun does.

This market is set up a bit like that. Manifold whales are much, much bigger than everyone else, to the tune of portfolios 1000 times bigger than the substantial amount Manifold gives new users to start. Minnows are numerous, but Isaac was willing to make a bet that there weren’t so many of them that the whales couldn’t buy 10,000 votes to offset each and every minnow participant.

There was some immediate controversy regarding when/how/if the market would be resolved, since with Isaac in control he could potentially choose a closing date for the market that favored him and Team Whales.This was eventually resolved by setting up a council of 3 relatively trusted Manifolders who would make those decisions independently of Isaac

and who settled on a semi-random “whenever the next bitcoin hash ending in 00 is mined” closing date criteria. 

And so the battlefield was set, ready for the ultimate confrontation between indoors-kid Davids and Goliaths.

The craziest thing about all this to me as a layman is that it probably should have worked. Manifold is active, but in a lot of ways, it’s a niche filled by people who are very serious about quant and who think in a very particular way. That makes for low numbers of participants in a lot of markets. From Isaac’s point of view, it would have looked very possible to win this, especially if he could corral most of the other whales.

For the majority of the market’s lifespan, the number of minnows hovered around 400-600 total participants, which meant Team Whales would have to pony up something like four to six million mana to bring home a win. This lead intensified when an enterprising minnow began to use a paid, $2-per-new-user tool to mass produce new minnows, and for the first time make it more likely than not that the minnows would win.

Isaac offered to buy out No holders with less than 2000 shares and cash them out as if the market had resolved no. If this went exactly his way, he would be able to chip off a large enough fraction of no votes to take down the minnow side. On top of that, Isaac bought the API keys that controlled the previously mentioned mass-recruited minnows from the Manifold user who held them and used that power to flip their votes to “Yes”.

Isaac also started to buy massive amounts of Mana. For other outsiders reading this, Mana is the play money that you use to back your predictions on Manifold; when you open an account, you get some for free. There are few ways to make small amounts of mana without any real-world money exchanging hands, but if you need a bulk amount fast the only real way to do it is either by winning a lot of long-shot markets really fast (which nobody can do on command) or by spending your own cash.

That’s a graphic representation of Isaac spending his own money to try and win the market. Notably, that’s Isaac spending money 

he can’t get back even if the market goes his way

 because Mana isn’t convertible back to cash (You can, however, donate Mana in your account 

to charity at a 1:1 ratio with its purchase price

). Unilaterally buying markets isn’t cheap, and at or near the top of that peak Isaac had $30,000 real-world, honest-to-god US dollars tied up in this market. 

Between this and the API tomfoolery, the minnows were at this point alternating between complaining about foul play and expressing concern for what was starting to look like a dangerous outcome for Isaac himself, noisily oscillating between both positions like an unbalanced washing machine basket.

If this all seems pretty bad to you, understand that you and I are on the same page. It seems that way to me too.

If you thought “Isaac is probably an uber-rich guy with unlimited funds to dump at this to prove a point”, the evidence points to you being wrong. With uber-energized minnows recruiting faster than ever on Reddit and on the Destiny livestream, Isaac himself was by the morning of 4/25 indicating he was out of money he could dump into the market, and despairing of his own chances to win. 

Manifold bets are cheaper the more they disagree with the current winning side, and up until this point the Minnows had been strategically holding back on purchases to keep Yes votes artificially expensive. Isaac admitting weakness put chum in the water, and the Minnows immediately swarmed with purchases and drove the market downward in a way it never recovered from.

At this drop, the chart started showing what close observers already knew; the market was effectively over, and the minnows had won.

OK, so, I want you to understand: Manifold is paying me for these articles. But at the same time, I’m writing them in my voice and name, so there’s a bit of an understanding that I can more or less say what I want. They don’t have to print what I write; that’s their call. But there’s some freedom that’s been asked for and granted on this project, and I’m utilizing it.

Mostly I’m using it to say the following: This story is interesting, this story is fun to write about, and this story is also horrifying and nobody should want stuff like this to happen.

The point of a prediction market is just that - to make predictions. It’s so you can log on and easily access the wisdom of the crowds to help you make decisions or to help you learn about the reality of a topic that might be so politically or culturally loaded that you otherwise would have difficulty getting a handle on it. 

your way of participating in that. Mana exists, near as I can tell, so that you feel like you have some skin in the game; it’s available for free, and buying a significant amount of it doesn’t necessarily cost a ton. Ideally, you put just enough value on Mana that it influences you to make careful, thoughtful bets.

What happened here wasn’t that. It’s been said a lot in the comments, but this story carries a lot of the flavor of gambling addiction - of someone who bets more than they want to lose, feels trapped, and keeps betting. And what we know about the story supports that; $29k does not seem to have been small-amount play money to Isaac.

I don’t understand every aspect of prediction markets. 

. I’m not an expert, but I do understand this: this is not what they are for. If you like these markets, this kind of betting is destructive and gets in the way of everything good you think they should do.

As far as Manifold goes, I’m only slightly more privy to their inner thoughts on this than you are. But from what I can tell, they agree with me at least that this was a bad event and it should be discouraged from happening again. First, and I think I approve of this a lot, they bailed out Isaac to the tune of $25k of the total 29k - an amount that I think means “here’s a penalty to make you not do it again, but also we don’t want to ruin your life to make a point”.

They also have been making some public statements in the comments that indicate they don’t like this and are trying to figure out how to prevent it while not constraining the markets too much:

To be clear once more, I’m sort of broadly allowed to say “Manifold is handling this poorly and I hate them” here. For the most part, they seem to instead be making the moves I’d want them to make - helping this be as small of a disaster as possible for the person most affected, and taking steps to prevent similar stuff while retaining community focus.

My understanding is that Manifold is working on a more comprehensive public statement about how they are working towards discouraging this kind of problem, and when they do I’d encourage them to link it.

(Manifold note: We did do this, and it’s available 

. We also put together a showcase of high-quality object-level markets that better represent Manifold 

In summary: This market was interesting, it’s interesting to talk about, and I hope I made it interesting to read about. Isaac made a huge bet on the idea that the whales had enough money to buy predictions in direct opposition to the mob, and lost. Looking just at that, this is a good thing; arguably, this was the most direct and conclusive way to show that wisdom-of-crowds can’t be taken down by wealth-of-the-few on Manifold at this point in time.

But looking at the social aspects of this (especially to the extent they concern Isaac) shows some dangers of the market; the bets are called bets for a reason. They 

 like bets, and Mana has been successful in 

. That’s necessary for the prediction markets to work at all, but it’s dangerous.

Manifold is taking steps to discourage the kind of markets and behavior that would hurt individuals that made bad decisions, but that’s only half of the picture; it’s important that the individuals themselves understand the psychological dangers of 

If a person can freak out because their important-to-them DnD character died and damage friendships over it (and they can, and do) then people can get too wrapped up in these markets. 

I’m traditionally a skeptic of ultra-quant solutions to things, and prediction markets fall under the heading. Writing both this piece and 

 has made me far, far more friendly to prediction markets than I was before; they are neat. They are interesting. I sort of get it a bit now. It would be a shame to limit that appeal by losing sight of the purpose of all this: to learn more, and make better decisions.

If you enjoyed this piece, be sure to check RC out on his 

, and takes advantage of inopportune times to mention he loves his wife.

Whales vs. Minnows and Pitfalls of Prediction Markets

dynomight.net/prediction-market-causation

We all want to make good decisions. But it’s hard because we aren’t sure what’s going to happen.

Like, say you want to know if CO₂ emissions will go up in 10 years. One of our best ideas is to have people bet. For example, I might wager my $4 that emissions will go up against your $1 that they will go down. If lots of people bet in a market, the final prices combine everyone’s wisdom and information. If bets were to stabilize at a 4:1 ratio, that suggests that the odds of emissions going up are 4 times larger than the odds of emissions going down, i.e.

, you often need more information. Say someone comes up with a treaty to reduce CO₂ emissions. If you want to know if the treaty will work, you can have people make 

 bets. For example, I might bet my $9 against your $1 with these rules:

If lots of people bet in a market, and prices settle at a 9:1 ratio, that suggests that emissions go up in 90% of the worlds where the treaty gets signed, i.e.

You can run a second market to get the odds of emissions going up, conditional on there 

So here’s our question: Say you do that and the market thinks that emissions are more likely to go up with the treaty than without. That is, say 

. Does that mean that the market thinks that signing the treaty will cause emissions to go up?

People worry about prediction markets for lots of reasons. Maybe someone will manipulate prices for political reasons. Maybe fees will distort prices. Maybe you’ll go Dr. Evil and bet that emissions will go up and then go emit a gazillion tons of CO₂ to ensure that you win. Valid concerns, but let’s ignore them and assume markets output “true” probabilities.

Now, what would explain the odds of emissions going up being higher with the treaty than without? The obvious explanation is that the market thinks the treaty will cause emissions to go up:

Totally plausible. But maybe the market thinks something else. Maybe the treaty does nothing but voters 

 it does something, so emissions going up would cause the treaty to be signed:

In this chain of events, the treaty acts as a kind of “emissions have gone up” award. Even though signing the treaty has no effect on emissions, the fact that it became law increases the odds that emissions have increased. You could still get the same probabilities as in a world where the treaty 

 (albeit with internet points instead of money)

    P[launch | don’t declare] = 5.4%.

 but you can find an implied price using (1) the market for 

 and (3) the ᴘᴏᴡᴇʀ ᴏꜰ ᴍᴀᴛʜ.

. The obvious way of looking at this would be that NATO declaring a no-fly zone would increase the odds of a nuclear launch:

NATO and Russian planes clash over Ukraine

That’s probably the right interpretation. But not 

. For example, do we really know the mettle of NATO leaders? It could be that declaring a no-fly zone has no direct impact on the odds of a launch, but the fact that NATO declares one reveals that NATO leaders have aggressive temperaments and are thus more likely to take 

 aggressive actions (note the first arrow points up):

This could also explain the current probabilities.

Is this starting to sound familiar? If we had a market, I’m confident investors would assign a vastly higher price for 

Is that because reading dynomight makes you more charming and good-looking? Sadly, no—it’s because such people are more likely to value prime-grade autodidactism.

Does this remind you of another issue in probability and statistics? Like, say, the 

 using prediction markets to predict if banned commenters should be reinstated. The idea is that if you get banned, you can write an argument that you will be good and then investors would bet on the odds that Scott would reinstate you if he read your justification, i.e.

This might not work. Suppose that Scott currently pays an assistant to pre-screen the justifications. If the assistant thinks there’s at least an 80% chance someone will be reinstated, they forward their justification to be read, while the rest are ignored. (The assistant gives 

 “ideal” predictions, in the sense that it’s impossible to be more accurate given the same information.)

This distorts the market. Assuming people know about the assistant, the market will give at least an 80% probability of reinstatement to 

, regardless of how bad it is. (You are guaranteed to make money over time by blindly taking any bet with odds lower than 80%. So people will always bid the odds up to 80% or higher.)

What went wrong? The problem is that the quality of the justification influences if it gets read:

             ↓          ↘  

This same distortion of probabilities happens with almost any selection based on the quality of the justifications, though I can’t figure out how to make the general case without some grungy calculations.

But here’s one example: Suppose that Scott looks at only those cases where the prediction market is above 25%. This will also distort the probabilities upward. Suppose that the market is currently at 30% but you think the true probability is 20%. Consider two cases:

That doesn’t seem fair, does it? In the situation where you’re right, the bet is likely to be canceled. So you’ll notice this and not bet. When people who think the probabilities are low drop out of the market, the probabilities get distorted upwards.

The solution to this is to read justifications chosen 

, without any consideration of the quality of the justification or market prices. This breaks the link from the justification to the event of being read:

             ↓             

Look at that: To make things work, all we needed to do was promise to our conditional prediction market that we will do the conditioning 

Where are we? Let’s review. Here’s the typical 

So far, this article has made this argument:

Basically: If you run a prediction market to predict correlations, you get correlations. If you run a prediction market to predict the outcome of a randomized trial, you get causality. But to incentivize people to predict the outcomes of a randomized trial you have to actually 

So that’s the problem. What can we do about it? There are several options.

One option is to think carefully about the system you’re studying. If the causal structure is OK (no reverse causality, no confounders), then you’re OK, and a conditional prediction market will give you what you want. But do be careful, because if you’re wrong you’ll get meaningless results.

There’s an interesting wrinkle here which is that I guess all that you 

 need is that people betting in the market 

 the causal structure is OK. It seems dangerous to try to exploit this, though.

We solved the problem with banned commenters by committing to choosing actions randomly. This option always works. But there’s a cruel irony: The whole point of prediction markets is to make good decisions. If you have to behave 

 to get good predictions, then what are we even doing here?

Fortunately, there’s a good (and well-known) alternative, which is to randomize decisions 

. You tell people: “I will roll a 20-sided die. If it comes up 1-19, everyone gets their money back and I do what I want. If it comes up 20, the bets activate and I decide what to do using a coinflip.”

What’s nice about this is that you can do whatever you want when 1-19 come up, including making your decisions using the market prices. But you must make decisions randomly 

, because if bets never activate, no one will waste their time betting in your market.

This is elegant. Oh, and by the way are you THE NSF or DARPA or THE NIH or A BILLIONAIRE WHO WANTS TO SPEND LOTS OF MONEY AND BRAG ABOUT HOW YOU ADVANCED THE STATE OF HUMAN KNOWLEDGE MORE THAN ALL THOSE OTHER LAME BILLIONAIRES WHO WOULDN’T KNOW A HIGH ROI IF IT HIT THEM IN THE FACE? Well how about this:

Still, I really hope that NATO doesn’t randomize the decision about a no-fly zone.

Consider again the case where people could bet on the odds of a commenter being reinstated. It would be nice to use a market to do filtering, i.e. only read the justifications that have a probability above 25%. But we saw that would distort the market.

There’s an alternative betting structure that could mostly solve this problem. The idea is that if the final market probability is above a threshold you always read the justification. If the final market probability is below a threshold, you only read the justification a fraction of the time, but you make the bets correspondingly larger. For example, we might agree to these rules:

Regardless of if the market is above or below 25%, 

, I will win $1 if I’m right that the person will be reinstated and you will win $4 if you’re right that they won’t. So this market will still incentivize people to bet their true beliefs.

Imagine a prediction market for the odds of a war conditional on who wins a presidential election. You couldn’t interpret this causally. For example, maybe candidate X is less likely to cause a war, but if it already looks like war is looming, then people might vote for candidate Y.

But we can’t randomly pick presidents. (That would interfere with our elections which are 

 just as bad as choosing a winner at random.)

 a market that only activates if the votes are very close to even, so that the outcome is 

This isn’t perfect, because an election being close might influence other variables. For example, imagine that candidate Y likes to undermine the legitimacy of elections. If there’s a clear result it won’t matter, but if X wins narrowly, riots would start and global adversaries might take advantage of the chaos to start wars. In a case like this, a “close call” prediction market might make it look like X is more likely to cause a war, even though X is better under most circumstances.

Still, this is a good idea—it’s not that far-fetched to just assume that things like that don’t happen. You can probably invent other markets with the same principle of “bet that activates if a natural experiment happens to occur”.

Robin Hanson has also suggested “fire the CEO” prediction markets where people bet on the future price of a company, conditional on a CEO being fired (or not). If such a market says the price is more likely to go down if the CEO is fired, that doesn’t necessarily mean the CEO is good because you might imagine something like this happening:

An obvious way to deal with this is to use time: Bet on the stock price 2 years from now, conditional on the CEO being fired in the next month.

This is still no guarantee because it only eliminates reverse causality, not hidden confounders. Maybe everyone knows the CEO is terrible, but hard to fire. But the company was just bought by some billionaire and there are weird rumors that the billionaire hates the CEO and wants to fire them and then drive the company into the ground out of spite:

If you learned that the CEO gets fired soon, you’d have to conclude that the billionaire really does hate them and plans to go on and destroy the company. So even though the CEO is terrible, you’ll have conditional probabilities that make it look like the CEO is good. Time is no magic bullet.

Controlled conditional prediction markets

Finally, one could add control variables to the prediction market. Say you want to know if you should fire the CEO, but you’re worried about things being confounded by a recession. Then you could run four prediction markets for the odds of the stock going up conditioned on each combination of the CEO being fired (or not) and there being a recession (or not). If the CEO truly is crap, you’d expect that