kjz avatar
resolved Mar 13
Will Manifold's developers agree with me that the dynamic parimutuel cost function should be changed?
Resolved
YES
I believe that the dynamic parimutuel betting system as formulated by Manifold (described in the site's technical guide) significantly disincentivizes accurate price setting and should be changed. I have written a blog post arguing this point, which I intend to publish shortly. This market will resolve to Yes if Manifold's developers publicly state an intent to change their pricing implementation, and this market contributed in some way to their decision to do so. This market will resolve to No if this does not happen within a month (either because I become convinced that my argument is wrong, or if I still believe in it but the developers do not agree with me that it is an issue). Fine print: This does not require a specific change to actually be implemented within the next month, as long as the developers publicly state an intention to implement a change. If a change is made to pricing that is unrelated to this market that does not count. In the unlikely event that a change is made but it is unclear whether it is or is not related to this market I will resolve to N/A. I have no inside information about Manifold nor any pre-existing relationship with its developers or with any other prediction markets. Feb 17, 6:50pm: Here's the argument https://kevin.zielnicki.com/2022/02/17/manifold/ Feb 19, 9:40pm: Here's a TLDR summary of the issue, which I still believe is significant enough to justify a change to Manifold's betting system: If market odds change substantially over time due to arrival of new information, earlier traders will not get the payouts they were expecting from the odds at the time of their bet. This is a common occurrence in interesting prediction markets, many of one can even expect will go to nearly 1 or 0 by the time they close (which is the worst case for this issue). This significantly distorts incentives for accurate pricing.

🏅 Top traders

#NameTotal profit
1
Unknown user avatar
John Beshir
Ṁ290
2
Unknown user avatar
S G
Ṁ103
3
Unknown user avatar
Pepe
Ṁ72
4
Unknown user avatar
Austin
Ṁ29
5
Unknown user avatar
Tobias Dänzer
Ṁ26

💬 Proven correct

jw avatar
jw
sold Ṁ1,153 of YES
Case in point. I bought at 42%...

💸 Best bet

A trader bought Ṁ3,000 YES from 84% to 95%
John Beshir made Ṁ190!
Sort by:
kjz avatar
Kevin Zielnicki
bought Ṁ500 of YES
Thank you @SG, excited for the change!
SG avatar
S G
bought Ṁ500 of YES
I think you can probably resolve this as YES. I intend to switch over the betting system for new binary markets to a CFMM (constant function market maker) sometime next week, and this market did play a role in leading us to switch.
kjz avatar
Kevin Zielnicki
bought Ṁ1 of YES
And re "I think -probably- if they say they currently expect to change it that should work too?" -- yes, I would consider this sufficient
kjz avatar
Kevin Zielnicki
bought Ṁ1 of YES
Or to put it another way, I want to separate "intent to change" from "intent to evaluate"
kjz avatar
Kevin Zielnicki
bought Ṁ1 of YES
@jbeshir I would also accept "intent to change" from the devs, I don't mean to introduce a higher level of commitment than originally set out in the market description. I just want to make sure I'm not overreading into a discussion of possible alternatives, if for example unmodified DPM is still considered a contender for market making in the long-term.
jbeshir avatar
John Beshir
bought Ṁ1 of YES
Just updating on that the wording in the above comment suggests it doesn't just need to be an intent to change, which could be e.g. conditional on testing working as expected with the existing algorithm a fallback position, but a "commitment", which is a higher bar than intent to change that I'm not at 97% about. I still think we're probably going to get something satisfying the market (I think -probably- if they say they currently expect to change it that should work too?), esp given that in the same channel we have a competing proposal to move to DPMM-with-sidepools which also is attempting to address this issue and I think should also qualify as a pricing implementation change, so DPMM as-is doesnt' seem to be the first fallback option here. But I'm less sure about how this is going to decide now.
PatrickDelaney avatar
Patrick Delaney
bought Ṁ10 of NO
I believe we can tell who just sold $3003 worth of YES based upon some simple forensic analysis. Based upon that person's record, I'm going further in on NO.
kjz avatar
Kevin Zielnicki
bought Ṁ1 of YES
From comments here & on discord, I'm inclined to think dev statements amount to an intent to change pricing in some way that would address the DPM issues discussed here, and thus the market should resolve yes (even if the exact form of the change may not yet be fully decided). @Austin or @SG could you confirm if this is a reasonable interpretation? The line I want to draw is between "we're evaluating different market makers, but may end up sticking with the current DPM implementation" which I would resolve to No (if still true at market close), while "we're evaluating different market makers, but are committed to choosing something other than the current DPM implementation" which I would resolve to Yes.
MattP avatar
Matt P
sold Ṁ6 of YES
Case in point why OP's argument was a good one - I put $10 down on this market when the implied probability was only 16%. My current expected payout for that? $24, for a profit of $14. ;P
MattP avatar
Matt P
bought Ṁ6 of YES
For those of us not in the know, what's the general info on CPMM?
jbeshir avatar
John Beshir
bought Ṁ3,000 of YES
https://discord.com/channels/915138780216823849/927671564177133618/951713160660930600 - CPMM implemented (on the dev site) with a statement of intent to bring it to the main site.
jw avatar
jw
sold Ṁ1,153 of YES
Case in point. I bought at 42%...
BoltonBailey avatar
Bolton Bailey
bought Ṁ1 of YES
Buying YES as a protest, I'm very disappointed to realize that the "Payout if YES" isn't actually a guarantee. This makes manifold no better than metaculus in my opinion, but hopefully it will change.
Austin avatar
Austin
bought Ṁ1 of YES
Quick update for y'all: we are still actively evaluating different market makers; currently excited about the Polymarket-style CPMM, though it doesn't scale well to multi-category or free response questions. Thanks for your patience on this front!
PatrickDelaney avatar
Patrick Delaney
bought Ṁ1 of NO
I still do not think that developers at Manifold are going to be able to respond to this request in the timeframe set up. However, I think it would be cool to allow the market maker to select which algorithm governs the bet (fixed for the duration of the market). Maybe there could be 2 or 3 different options so it's not too complex, feature different emoji/symbols for the type of market algorithm, this would be a cool way to introduce people intuitively to different styles of market algorithms.
Accountdeletionrequested avatar
Account deletion requested
bought Ṁ1 of YES
One more question where participating with current payout is pointless: https://manifold.markets/ValentinManes/will-george-rr-martin-publish-an-as
PatrickDelaney avatar
Patrick Delaney
bought Ṁ5 of NO
Set up this market referencing this DPCF market even though my bet here on the DPCF market was no. https://manifold.markets/PatrickDelaney/will-russia-invade-ukraine-in-march
Pepe avatar
Pepe
bought Ṁ10 of YES
Another way of stating this is that the *best* ratio of YES odds and NO odds in this kind of market is 5.8. You can calculate this odds ratio gap as a function of the probabilities that YES and NO will reach 1 early, or some more complicated model. I recall someone at the forecasting meetup saying a typical e.g. sports betting market has a bid-ask spread of 2%. I'm guessing DPM has a larger gap than CLOB down to about a 5% chance of the market learning the outcome early.
Pepe avatar
Pepe
bought Ṁ10 of YES
@SG and some others have stated this as a problem with the interaction between how the math works and how information is displayed to the user. I disagree. DPM generates an enormous bid-ask spread which makes markets "will x happen by y date" markets very distorted. Consider the most popular market https://manifold.markets/Duncan/will-russia-invade-ukraine-before-t Assuming shares go to 0 or 1 before resolution, the current implied probability of buying YES is 62.5%, while NO is 21.8%. If this market resolves NO it's guaranteed to go to 0. If it resolves YES it's also quite likely to go to 1, unless the creator somehow sees the event happen before anyone else. This is not a volume issue. A market at 50% will give payoffs of 70.7% and 29.2% - a gap of 40% - no matter the volume. So how does the probability move at all? Only very confident bettors who think there's an extreme probability will participate, and we see the ratio between the extreme YES and extreme NO groups. That's some info about the distribution of beliefs and hence "true" probability, but it sure ain't the mean or median.
orrukje avatar
orrukje
bought Ṁ1 of YES
Thanks for all of the helpful comments on this question. @SG could you say a little more about what you see as the disadvantages of other options, especially Hanson-style AMMs? I find DPM hard to reason about / anticipate and would feel better knowing it was the last bad option for what Manifold is trying to do :)
PatrickDelaney avatar
Patrick Delaney
bought Ṁ10 of NO
I don't think there will be significant movement on the part of Manifold per your bet for three reasons: 1) On their about page I recall Manifold stating that their working hypothesis is that over time more skilled market makers will implicitly learn the flaws of this algorithm and will take on practices such as, ending the bet prior to the game beginning. I believe this is the stated working hypothesis right now, and 2) it takes a lot of resources to change code and align a team away from that. Incentivizing market makers to improve their market making capabilities with prompts and checkboxes is much simpler than changing the entire betting algorithm. 3) Dynamic models, flawed and unfair as they may be, will incentivize a lot of, "edge density," at the point of the end of the bets, and as a startup, or new platform rather, their main objective right now is increasing traffic and engagement. They don't have a, "fairness betting" dashboard, they are looking at firebase usage and trying to get users going. Basically what you are asking for is a feature request and in the reality of software development that is extremely likely to be tossed into the ice box, which means a non-commitment toward fixing, not a positive commitment.
SG avatar
S G
bought Ṁ1 of YES
Bet on which system we'll be using (and feel free to suggest alternatives): https://manifold.markets/ManifoldMarkets/which-betting-system-will-manifold
SG avatar
S G
sold Ṁ256 of NO
I'll continue to think about this. It's not that I don't see the problem here; it's just that all of the alternative mechanisms (order book, uniswap-style AMM, Hanson-style AMM) have problems of their own too. DPM works very nicely with our user-created market model but does have some very serious downsides.
VivekHebbar avatar
Vivek Hebbar
bought Ṁ1 of YES
@SG I don't think it is a rare issue; lots of bets go near 0% or 100% before trading closes.
VivekHebbar avatar
Vivek Hebbar
bought Ṁ9 of YES
Agree that the cost function should be looked at -- there are many markets where correcting a low probability by buying and holding yes is punished, causing 1) Naive traders to lose money in expectation even when they were correct about the probability being too low and 2) Knowledgable traders to refrain from correcting probabilities in these cases. For instance, if I thought this market should be at 40%, it would probably still be unprofitable to buy a large amount of yes. I think the current system also punishes putting ante even if the probability is guessed correctly.
kjz avatar
Kevin Zielnicki
bought Ṁ20 of YES
Thanks for the reply @SG! I do still remain convinced that this is a significant issue, so buying some shares while the price is low in hopes that you change your mind. Another example of where this distorts incentives is the popular Ukraine invasion market https://manifold.markets/Duncan/will-russia-invade-ukraine-before-t -- we can anticipate this market eventually going to nearly 1 or 0 by the time it closes. As to why it hasn't been a noticeable issue yet, I suspect that traders have been acting against their rational interests due to the novelty of the market and fact that we have free play money.
hamnox avatar
Em of the Night ☑️
sold Ṁ8 of NO
I doubt they will change the algorithm, but it would be good to make it clearer early on how other people's bets or sales will affect your shares. I understand dynamic market behavior is not a trivial concept to visualize accessibly, tho, it's a fairly hard ask.
MattP avatar
Matt P
bought Ṁ10 of YES
@SG I don't see how you can maintain the current structure without at least changing the way the interface presents expected payouts. The whole point of Manifold is to make betting markets more accessible, right? An interface that effectively lies to the bettor is going to work against this - and as Kevin describes in his blog post, the interface does in fact lie regarding expected payouts. Intuitively, someone who places a "Yes" bet when the implied probability is 20%, bidding it up to 25%, should expect somewhere between a 4x or 5x payout if they are correct. That's what the interface tells them, and it's the whole point of placing long shot bets. I don't know if the pricing algorithm needs to change or if the interface needs to be updated, but one of the two has to change. They are currently very much at odds.
JamesGrugett avatar
James
bought Ṁ1 of NO
One note is that market creators can cater to traders' preferences by coming up with markets that don't have this problem. I.e. they close early, or have different criteria so there's no such opportunity for news traders. A question I have is how different the returns are if there is more buying of both YES and NO at different probabilities. Like, say in the example, $1000 was bought around 30% after Alice bought in. (e.g. Dan and Edgar alternating buying $10 and $25 of YES and NO). How does that affect Alice's return?
SG avatar
S G
bought Ṁ1 of NO
@RobertMcIntyre, if there's a slow drip of news, betting volume tends to be a lot more balanced. And the case @kjz picked (where the initial probability if very low and then there's news decisively pointing to YES) is the very worst; if there probability is at 80% before the market-resolving news comes in, the payouts are much more in line with expectations.
RLMgold avatar
Robert McIntyre (gold)
bought Ṁ1 of YES
@SG but what about the "slow drip" case? That type of market is what interests me most and it seems to have a similar problem, no?
SG avatar
S G
bought Ṁ200 of NO
Thanks again for taking the time to write up your thoughts! In brief, the the failure case you're describing is when a piece of news arrives which leads to the outcome being decided before the market closes, causing traders to bid up the price (from a low probability) without any offsetting bets on the other side, which in turn, causes earlier traders to not make their odds. I am aware of this issue but think in practice it's not that bad: Most events that would fall into this category are like sports games or elections and have known resolution dates that the market creator can plan for in advance by setting the appropriate market close time. And empirically on this site, it hasn't been much of a problem. Another option we've thought of to mitigate this problem is to allow the market creator to adjust the close time post hoc to exclude trades made after a market-resolving piece of news is announced.
RLMgold avatar
Robert McIntyre (gold)
bought Ṁ150 of YES
This feels like it's pretty important for the algorithm, since markets where you know the outcome before it closes are pretty common, in fact I'd say they should be considered the base case since they allow for the most interesting and flexible uses of prediction markets.
Austin avatar
Austin
bought Ṁ1 of NO
Thanks for creating this market and writing up your thoughts! Without stating an intention to change: our DPM mechanism is quite novel; we've already updated it a few times to be more accurate. I'm sharing your blog post with @SG (the architect behind our DPM system). For those interested: you can play around with the DPM mechanism here: https://manifold.markets/simulator If you're interested in this topic, you can always come chat with us on our dynamic-parimutuel Discord channel! https://discord.gg/SeWCkHc7H4