Will Manifold's developers agree with me that the dynamic parimutuel cost function should be changed?
Resolved
YESI believe that the dynamic parimutuel betting system as formulated by Manifold (described in the site's technical guide) significantly disincentivizes accurate price setting and should be changed. I have written a blog post arguing this point, which I intend to publish shortly. This market will resolve to Yes if Manifold's developers publicly state an intent to change their pricing implementation, and this market contributed in some way to their decision to do so. This market will resolve to No if this does not happen within a month (either because I become convinced that my argument is wrong, or if I still believe in it but the developers do not agree with me that it is an issue).
Fine print: This does not require a specific change to actually be implemented within the next month, as long as the developers publicly state an intention to implement a change. If a change is made to pricing that is unrelated to this market that does not count. In the unlikely event that a change is made but it is unclear whether it is or is not related to this market I will resolve to N/A. I have no inside information about Manifold nor any pre-existing relationship with its developers or with any other prediction markets.
Feb 17, 6:50pm: Here's the argument https://kevin.zielnicki.com/2022/02/17/manifold/
Feb 19, 9:40pm: Here's a TLDR summary of the issue, which I still believe is significant enough to justify a change to Manifold's betting system: If market odds change substantially over time due to arrival of new information, earlier traders will not get the payouts they were expecting from the odds at the time of their bet. This is a common occurrence in interesting prediction markets, many of one can even expect will go to nearly 1 or 0 by the time they close (which is the worst case for this issue). This significantly distorts incentives for accurate pricing.
💬 Proven correct
jw sold M$1,153 of YES3 months ago
Case in point. I bought at 42%...
jw made M$145!
Kevin Zielnicki bought M$500 of YES3 months ago
Thank you @SG, excited for the change!
S G bought M$500 of YES3 months ago
I think you can probably resolve this as YES. I intend to switch over the betting system for new binary markets to a CFMM (constant function market maker) sometime next week, and this market did play a role in leading us to switch.
Kevin Zielnicki bought M$1 of YES3 months ago
And re "I think -probably- if they say they currently expect to change it that should work too?" -- yes, I would consider this sufficient
Kevin Zielnicki bought M$1 of YES3 months ago
Or to put it another way, I want to separate "intent to change" from "intent to evaluate"
Kevin Zielnicki bought M$1 of YES3 months ago
@jbeshir I would also accept "intent to change" from the devs, I don't mean to introduce a higher level of commitment than originally set out in the market description. I just want to make sure I'm not overreading into a discussion of possible alternatives, if for example unmodified DPM is still considered a contender for market making in the long-term.
John Beshir bought M$1 of YES3 months ago
Just updating on that the wording in the above comment suggests it doesn't just need to be an intent to change, which could be e.g. conditional on testing working as expected with the existing algorithm a fallback position, but a "commitment", which is a higher bar than intent to change that I'm not at 97% about.
I still think we're probably going to get something satisfying the market (I think -probably- if they say they currently expect to change it that should work too?), esp given that in the same channel we have a competing proposal to move to DPMM-with-sidepools which also is attempting to address this issue and I think should also qualify as a pricing implementation change, so DPMM as-is doesnt' seem to be the first fallback option here.
But I'm less sure about how this is going to decide now.
Patrick Delaney bought M$10 of NO3 months ago
I believe we can tell who just sold $3003 worth of YES based upon some simple forensic analysis. Based upon that person's record, I'm going further in on NO.
Kevin Zielnicki bought M$1 of YES3 months ago
From comments here & on discord, I'm inclined to think dev statements amount to an intent to change pricing in some way that would address the DPM issues discussed here, and thus the market should resolve yes (even if the exact form of the change may not yet be fully decided). @Austin or @SG could you confirm if this is a reasonable interpretation?
The line I want to draw is between "we're evaluating different market makers, but may end up sticking with the current DPM implementation" which I would resolve to No (if still true at market close), while "we're evaluating different market makers, but are committed to choosing something other than the current DPM implementation" which I would resolve to Yes.
Matt P sold M$6 of YES3 months ago
Case in point why OP's argument was a good one - I put $10 down on this market when the implied probability was only 16%. My current expected payout for that? $24, for a profit of $14. ;P
Matt P bought M$6 of YES3 months ago
For those of us not in the know, what's the general info on CPMM?
John Beshir bought M$3,000 of YES3 months ago
https://discord.com/channels/915138780216823849/927671564177133618/951713160660930600 - CPMM implemented (on the dev site) with a statement of intent to bring it to the main site.
jw sold M$1,153 of YES3 months ago
Case in point. I bought at 42%...
Bolton Bailey bought M$1 of YES3 months ago
Buying YES as a protest, I'm very disappointed to realize that the "Payout if YES" isn't actually a guarantee. This makes manifold no better than metaculus in my opinion, but hopefully it will change.
Austin bought M$1 of YES3 months ago
Quick update for y'all: we are still actively evaluating different market makers; currently excited about the Polymarket-style CPMM, though it doesn't scale well to multi-category or free response questions. Thanks for your patience on this front!
Patrick Delaney bought M$1 of NO3 months ago
I still do not think that developers at Manifold are going to be able to respond to this request in the timeframe set up. However, I think it would be cool to allow the market maker to select which algorithm governs the bet (fixed for the duration of the market). Maybe there could be 2 or 3 different options so it's not too complex, feature different emoji/symbols for the type of market algorithm, this would be a cool way to introduce people intuitively to different styles of market algorithms.
M bought M$1 of YES3 months ago
One more question where participating with current payout is pointless: https://manifold.markets/ValentinManes/will-george-rr-martin-publish-an-as
Patrick Delaney bought M$5 of NO3 months ago
Set up this market referencing this DPCF market even though my bet here on the DPCF market was no. https://manifold.markets/PatrickDelaney/will-russia-invade-ukraine-in-march
Pepe bought M$10 of YES3 months ago
Another way of stating this is that the *best* ratio of YES odds and NO odds in this kind of market is 5.8. You can calculate this odds ratio gap as a function of the probabilities that YES and NO will reach 1 early, or some more complicated model. I recall someone at the forecasting meetup saying a typical e.g. sports betting market has a bid-ask spread of 2%. I'm guessing DPM has a larger gap than CLOB down to about a 5% chance of the market learning the outcome early.
Pepe bought M$10 of YES3 months ago
@SG and some others have stated this as a problem with the interaction between how the math works and how information is displayed to the user. I disagree. DPM generates an enormous bid-ask spread which makes markets "will x happen by y date" markets very distorted. Consider the most popular market
https://manifold.markets/Duncan/will-russia-invade-ukraine-before-t
Assuming shares go to 0 or 1 before resolution, the current implied probability of buying YES is 62.5%, while NO is 21.8%. If this market resolves NO it's guaranteed to go to 0. If it resolves YES it's also quite likely to go to 1, unless the creator somehow sees the event happen before anyone else.
This is not a volume issue. A market at 50% will give payoffs of 70.7% and 29.2% - a gap of 40% - no matter the volume. So how does the probability move at all? Only very confident bettors who think there's an extreme probability will participate, and we see the ratio between the extreme YES and extreme NO groups. That's some info about the distribution of beliefs and hence "true" probability, but it sure ain't the mean or median.
orrukje bought M$1 of YES3 months ago
Thanks for all of the helpful comments on this question. @SG could you say a little more about what you see as the disadvantages of other options, especially Hanson-style AMMs? I find DPM hard to reason about / anticipate and would feel better knowing it was the last bad option for what Manifold is trying to do :)
Patrick Delaney bought M$10 of NO3 months ago
I don't think there will be significant movement on the part of Manifold per your bet for three reasons: 1) On their about page I recall Manifold stating that their working hypothesis is that over time more skilled market makers will implicitly learn the flaws of this algorithm and will take on practices such as, ending the bet prior to the game beginning. I believe this is the stated working hypothesis right now, and 2) it takes a lot of resources to change code and align a team away from that. Incentivizing market makers to improve their market making capabilities with prompts and checkboxes is much simpler than changing the entire betting algorithm. 3) Dynamic models, flawed and unfair as they may be, will incentivize a lot of, "edge density," at the point of the end of the bets, and as a startup, or new platform rather, their main objective right now is increasing traffic and engagement. They don't have a, "fairness betting" dashboard, they are looking at firebase usage and trying to get users going. Basically what you are asking for is a feature request and in the reality of software development that is extremely likely to be tossed into the ice box, which means a non-commitment toward fixing, not a positive commitment.
S G bought M$1 of YES3 months ago
Bet on which system we'll be using (and feel free to suggest alternatives): https://manifold.markets/ManifoldMarkets/which-betting-system-will-manifold
S G sold M$256 of NO3 months ago
I'll continue to think about this. It's not that I don't see the problem here; it's just that all of the alternative mechanisms (order book, uniswap-style AMM, Hanson-style AMM) have problems of their own too. DPM works very nicely with our user-created market model but does have some very serious downsides.
Vivek Hebbar bought M$1 of YES3 months ago
@SG I don't think it is a rare issue; lots of bets go near 0% or 100% before trading closes.
Vivek Hebbar bought M$9 of YES3 months ago
Agree that the cost function should be looked at -- there are many markets where correcting a low probability by buying and holding yes is punished, causing 1) Naive traders to lose money in expectation even when they were correct about the probability being too low and 2) Knowledgable traders to refrain from correcting probabilities in these cases.
For instance, if I thought this market should be at 40%, it would probably still be unprofitable to buy a large amount of yes. I think the current system also punishes putting ante even if the probability is guessed correctly.
Kevin Zielnicki bought M$20 of YES3 months ago
Thanks for the reply @SG! I do still remain convinced that this is a significant issue, so buying some shares while the price is low in hopes that you change your mind. Another example of where this distorts incentives is the popular Ukraine invasion market https://manifold.markets/Duncan/will-russia-invade-ukraine-before-t -- we can anticipate this market eventually going to nearly 1 or 0 by the time it closes. As to why it hasn't been a noticeable issue yet, I suspect that traders have been acting against their rational interests due to the novelty of the market and fact that we have free play money.
Em of the Night sold M$8 of NO3 months ago
I doubt they will change the algorithm, but it would be good to make it clearer early on how other people's bets or sales will affect your shares. I understand dynamic market behavior is not a trivial concept to visualize accessibly, tho, it's a fairly hard ask.
Matt P bought M$10 of YES3 months ago
@SG I don't see how you can maintain the current structure without at least changing the way the interface presents expected payouts. The whole point of Manifold is to make betting markets more accessible, right? An interface that effectively lies to the bettor is going to work against this - and as Kevin describes in his blog post, the interface does in fact lie regarding expected payouts.
Intuitively, someone who places a "Yes" bet when the implied probability is 20%, bidding it up to 25%, should expect somewhere between a 4x or 5x payout if they are correct. That's what the interface tells them, and it's the whole point of placing long shot bets. I don't know if the pricing algorithm needs to change or if the interface needs to be updated, but one of the two has to change. They are currently very much at odds.
James Grugett bought M$1 of NO3 months ago
One note is that market creators can cater to traders' preferences by coming up with markets that don't have this problem. I.e. they close early, or have different criteria so there's no such opportunity for news traders.
A question I have is how different the returns are if there is more buying of both YES and NO at different probabilities. Like, say in the example, $1000 was bought around 30% after Alice bought in. (e.g. Dan and Edgar alternating buying $10 and $25 of YES and NO). How does that affect Alice's return?
S G bought M$1 of NO3 months ago
@RobertMcIntyre, if there's a slow drip of news, betting volume tends to be a lot more balanced.
And the case @kjz picked (where the initial probability if very low and then there's news decisively pointing to YES) is the very worst; if there probability is at 80% before the market-resolving news comes in, the payouts are much more in line with expectations.
Robert McIntyre (gold) bought M$1 of YES3 months ago
@SG but what about the "slow drip" case? That type of market is what interests me most and it seems to have a similar problem, no?
S G bought M$200 of NO3 months ago
Thanks again for taking the time to write up your thoughts!
In brief, the the failure case you're describing is when a piece of news arrives which leads to the outcome being decided before the market closes, causing traders to bid up the price (from a low probability) without any offsetting bets on the other side, which in turn, causes earlier traders to not make their odds.
I am aware of this issue but think in practice it's not that bad: Most events that would fall into this category are like sports games or elections and have known resolution dates that the market creator can plan for in advance by setting the appropriate market close time. And empirically on this site, it hasn't been much of a problem.
Another option we've thought of to mitigate this problem is to allow the market creator to adjust the close time post hoc to exclude trades made after a market-resolving piece of news is announced.
Robert McIntyre (gold) bought M$150 of YES3 months ago
This feels like it's pretty important for the algorithm, since markets where you know the outcome before it closes are pretty common, in fact I'd say they should be considered the base case since they allow for the most interesting and flexible uses of prediction markets.
Austin bought M$1 of NO3 months ago
Thanks for creating this market and writing up your thoughts! Without stating an intention to change: our DPM mechanism is quite novel; we've already updated it a few times to be more accurate. I'm sharing your blog post with @SG (the architect behind our DPM system).
For those interested: you can play around with the DPM mechanism here: https://manifold.markets/simulator
If you're interested in this topic, you can always come chat with us on our dynamic-parimutuel Discord channel! https://discord.gg/SeWCkHc7H4