I'll allow any reasonable type of scoring to demonstrate this, such as the ones described here. (With appropriate corrections for multiple-comparisons bias if someone tries a bunch of different metrics.)
Proving causality would be next-to-impossible for something like this, so I'm only going to require that there be enough of a difference between the score before the change and the score afterwards that it seems plausible this was the cause.
1,000People are also trading
@ian I think I should wait for the new system to get settled in and at least a few dozen real money markets to resolve.
I think in effect you can kind of conceive of Manifold as a real-money market, given that the money can be donated to charity. If you subtract what you profit and donate via Manifold from regular donations you make from income, essentially you have made that much capital back. Plus you can add capital to get a big enough balance such that your bets might correspond to this amount. A nice efficiency with it going to charity is that it's essentially making real-world profit without the tax @jack mentioned.
I think this feels nice overall: your predictions are a utility to society (by providing it with information), and if you do well at it your profit goes to charity and offsets what you would normally donate from income. If you lose money, no worries! In the case where you put money in to create a decent balance in the first place, that'll still end up going to charity.
I think the main difference is that because you can't freely deposit and withdraw funds, most people do not treat it like real money.
We can observe that the typical amounts being traded are orders of magnitude smaller in USD terms than real-money prediction markets, which I think is largely driven by this difference. Examples to show how big the difference is:
https://manifold.markets/NcyRocks/will-a-democrat-win-the-2022-us-sen-3d2432ba6d79 liquidity = 20 USD, volume = 300 USD
https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-georgia liquidity = 40k USD, volume = 200k USD
https://manifold.markets/jack/will-the-fed-raise-the-fed-funds-ra-e142ec5d539b liquidity = 8 USD, volume = 25 USD
https://kalshi.com/events/FED-22DEC/markets/FED-22DEC-T4.50 liquidity = 10k USD, volume = 200k USD
Also, while it's true that profits aren't taxed on Manifold, buying mana and then donating doesn't get you any tax-deduction compared to directly donating - so for many people it's more tax-efficient to directly donate. (I think this is a much more minor consideration compared to the first.)
@IsaacKing of course, although I think there's a decent overlap between strong forecasters on prediction market platforms and people that donate to begin with. One positive spin is that if regular low-stakes forecasting becomes more mainstream through a platform like Manifold, it could act as an introduction and catalyst for some to make giving a habit.
@jack I know it's too small of a sample, but your recent analysis of the midterms suggests that maybe volume doesn't improve accuracy linearly. Metaculus rewards forecasters essentially in terms of reputation, and that seems to be a good enough incentive to attract good forecasters for now. I'd be interested in your thoughts on this given your activity across all platforms.
I hadn't thought about the tax-deduction thing. Because you know the money you're given by Manifold at the outset is going to charity regardless, I guess that reduces your incentive to win to zero again anyway.
Allowing real-money withdrawals from Manifold a terrible idea in so many ways. First, there is all of the legal implications since it would make it a betting/gambling site. Second, this would make Manifold lose one of its major appeals: that there is no actual stakes, since it is just play money. Third, it introduces very strong incentives to break and exploit the site, e.g. deliberately misresolving polls, setting up fake accounts, hacking, etc., since now actual money can be gained from this. Fourth, users would no longer be given free credits when they first join; even if they were prevented from withdrawing it, they could just create a bunch of new accounts and transfer those funds to a single account and withdraw from there; no free credits on join means new users have to spend money. Fifth, what happens to existing virtual currency? It feels unfair that users who haven't put money in to buy virtual currency can pull out money, which effectively came from users who did put real money in.
In order to resolve all these problems, it would be easier to just make a new real-money betting site from scratch, rather than try an change Manifold into it.
I think real-money markets may tend to get a much larger amount of trading (both in user count and $ terms), which is likely to improve accuracy. But for an equal amount of trading, I think play-money markets have some structural advantages e.g. profit incentives not being distorted by taxes, while perhaps also having some structural disadvantages e.g. people caring slightly more about topping the leaderboard vs maximizing EV.
@jack Ah, very good point. For the purposes of this market, I will not control by volume. But that would be a neat thing to look into as well.
Taking to the extreme, I'd love to see a real-money fork of Manifold that copies all its markets and then compare its accuracy to the play money Manifold.
@jack "people caring slightly more about topping the leaderboard vs maximizing EV"
This absolutely distorts my own betting (*cough* sorry, "predicting") behavior. Maybe not in the long run (since maximizing EV should still win out there), but for short-term events like the midterms tourney I'll absolutely bet overly confidently on the outcomes I think more likely, as I think that's more likely to result in big enough wins to hit leaderboard than going pure EV (though it's also much more likely to bust - somewhat of an "all or nothing" approach for sure).
@IsaacKing The amount of money that people have to spend in real life varies in ways that do not correlate strongly with that person's ability to predict the world. certainly somewhat, but the majority of monetary inequality comes from momentum. because of this, introducing real money prediction would incentivize contribution from big money but reduce the incentive for a smaller money to participate. I personally do not participate in real money prediction markets because I am much less risk tolerant with real money due to my financial situation.
For the record, I vote against real-money withdrawals. I really like Manifold and think that the experience would be worse if I could (try to) make money from it. I would be less playful, more worried about wrong bets.
There are probably many regulatory hurdles. I would be sad if Manifold became unavailable in certain jurisdictions due to trading or gambling regulations.
Finally, the playful nature of Manifold distinguishes it from the likes of PolyMarket, Kalshi, ...