Manifold's current implementation of numeric markets allows users to bet on a scalar value within a numerical range. Under the hood, it operates using the same mechanism as a binary market, with the low and high points of the range mapped to 0% and 100%, respectively.
Many of our users have reported being confused by numeric markets, in particular by the concept of HIGHER and LOWER shares, the fact that you can only bet within a given range, and the low payouts that result if there isn't significant price action. Given the added complexity to our platform, we're wondering if we should scrap this type of market entirely.
This market resolves YES is we remove numeric markets as a choice from the "create a market" page at any point within the next 7 days. (We almost certainly won't remove or delete existing numeric markets, regardless of what we decide, at least until they've resolved.)
You can influence this market by posting your thoughts below!
Resolution
I have decided to remove numerical and multiple choice markets from the create page. (This decision is not necessarily final and may be reversed.)
While there is clearly demand for markets on scalar values, our current implementation is a poor experience for everyone and a source of confusion for newer users. Removing numerical markets as an option pushes users to create binary markets, which is a much better experience for traders (as evidenced by the higher trading volumes and fewer complaints). We are actively considering reworking numerical markets and hopefully will introduce a new and improved form in not too long.
đ Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | áš251 | |
| 2 | áš92 | |
| 3 | áš28 | |
| 4 | áš20 | |
| 5 | áš19 |
Given the argument that range markets are less exciting because typical winnings are lower, I'd like to point out that the winning answer on my last free response market had a 75x return. They're obviously higher variance on average than binary markets, so maybe give them some love.
Breaking my thoughts up into a few parts:
The current implementation is confusing, very rarely profitable to bet on, and because savvy users largely avoid betting on it, the results are uninformative.
A good system for numeric markets would be a very desirable feature; there's a ton of markets that are naturally numeric, and even where unnatural, numbers are a better kludge than binary probability.
If a good numeric system is introduced, there will be confusion due to the mix of old system numeric markets and new system numeric markets. This confusion will be proportionate to how many (active) numeric markets still exist on the old system.
Disabling new numeric markets until a good system can be found would mean that there are fewer numeric markets, especially active ones, which could cause that confusion.
So I think it is a very good idea to disable it until a better system is found.
Because existing numeric markets are just reskinned binary markets I think it will be easy to get them out of the way when new numeric markets are introduced.
I think a potentially better version of numeric markets would involve betting on whether the outcome will be in certain ranges (i.e. capture distributional information), as opposed to just betting on the expected value of the outcome (like numeric markets today). Think of multiple binary markets on whether the outcome will be between up to 100, up to 200, up to 300, etc. (Yes, you can already do this ad-hoc by making a couple of yes/no markets, but the user experience can be vastly improved by making it an actual feature.)
This addresses a lot of the downsides of numeric markets today - it's easier to understand, and it provides better liquidity and exciting high payouts (if you set the right thresholds). It also captures more information, by showing a prediction of the distribution of possible outcomes.
The thresholds can be a handful chosen by the author, and then traders place individual bets on these binary markets (like Kalshi). Or you can have an automatic bucketization system and have the market automatically translate user bets into yes/no bets on the buckets (this is more like numeric markets v1, but obviously I want a better system for inputting bets, and this would probably end up kind of like Metaculus numeric markets).
@jack I like this approach because it means that Manifold fixed multiple choice markets which I think have a lot of potential for excitement.
Myself, I created a numeric market, and I am annoyed that it doesn't work as well as expected. In particular, I had no idea at all that changing the upper and lower limit has a significant impact on liquidity. I would not miss them after removal.
I really don't want to go back to people doing numeric markets by translating numbers into percentages by hand. I think you should first at least try to make the user experience more comprehensible before writing them off entirely.
@MichaelWheatley Or worse, abusing multiple-choice markets with numeric ranges. (Thus making it very frustrating to bet without accidentally placing an implicit probability of 0 on some outcomes.)
I wouldn't miss them much. They are among the least exciting markets to bet on.
I do think the site should have some form of numeric markets, though. It's a useful thing to be able to ask. The current mechanism is just kinda boring (plus it doesn't give any info about the distribution).
@MattP A note of caution: I refuse to be on metaculus numeric markets now, because it's so hard to enter in a correct distribution. (Correct in the sense of "an accurate reflection of what I believe".)
When I bet, I like to think of it as providing a single bit of information. That makes it tractable.
The low liquidity, in the region of interest, is a problem though. Perhaps if the creator could specify a distribution? And then bets would move along in increments of equal probability. (The support for log scale is a specific case of this!)
In that scenario, markets can become "stale" when the actual distribution begins to differ too strongly from what the creator had in mind. But then, exactly the same thing is true of binary markets, where some questions get much less interesting over time---the solution is just to make new markets that ask better questions.
In my hopelessly optimistic mind, this even merges with the proposal from @ian. Rather than requiring payment (or a large account) to create numeric markets, one can just make creating them somewhat more time-intensive.
@ScottLawrence yeah - to be clear, I'm not sure if there is a good way to do the "distribution" style - I just wish there was.
@MattP Actually, as you just mentioned, better tools for mass-entering limit orders gets you very close to distributional information, right?
@ScottLawrence No, limit orders do not really provide distributional information. The market price is fundamentally a reflection of the expected value (average). The market should generally be willing to put limit orders around the correct expected value - there is no reason to put limit orders around the possible/likely outcomes.
@jack Not sure about this. In practice, when I go look at the order books of various markets, the limit orders are clearly providing distributional information---they're certainly not concentrated around any expected value. There are many plausible reasons why people do this (hoping to take dumb money, planning for future information, realizing that it's not going to be possible to remove limit order instantaneously, not wanting to place a bet until the expected value is high enough), but it seems relevant that it happens in practice. It's also much more obvious on the salem center markets, indicating that this effect gets stronger with high liquidity, not weaker.
@ScottLawrence Just to clarify what distributional information we're talking about (which I think is also what you meant since you mentioned Metaculus numeric markets) - what I mean is like seeing whether the outcome is say normal, bimodal, left-skewed, etc. Take a question how late will my flight be? Say the answer is average delay of 20 minutes. I don't care about that as much as what percent of flights are delayed say >1hour.
I think the fact that limit orders have a wide distribution is a completely different thing. It's because different bettors have different ideas of what they think the right price is, how big a spread they want, how much risk they are willing to take, etc. But note that even on a yes/no market which has only the two possible resolutions yes/no, people still use limit orders, very much like they do on numeric markets that can resolve at any value. What I'm saying is you can't in general infer from the limit orders information about which outcomes are likely or even possible. The distribution of outcomes does affect where people are willing to put limit orders, because it affects how much risk you are taking at different price points, but it's extremely difficult to read the limit order book and learn anything about the distribution.
I posted another comment above on how numeric markets can support distributional information by bucketization - making several yes/no questions on will the outcome be in some range. And I think this also helps with the user engagement issues of numeric markets current implementation.
@jack the buckets are a nice idea. I worry it'll make liquidity problems worse (trying to pack too much info into m$100 of liquidity), but I'm not at all sure.
When I read the order book, I see a lot of statements of the form "no info will come out soon which will cause this market to rationally exceed thus-and-such". That's not distributional itself, but it becomes distributional in the limit where limit orders are never retracted.
But this is nitpicking. I agree that you can't reliably read out The Distribution.
No, log scales would not be that simple on a yes/no. Maybe implement it like Metaculus?
I don't know why everyone is so focus on how complex/intuitive they are. That is something many elements on our site need improvement on and I don't think really plays into the decision on whether they should be kept or removed.
The reason they should be removed is because they get very little engagement.
1. It's a bad experience for creators as their markets get 1-3 people betting on them max.
2. They clog up the market feed for users who aren't interested in betting on numeric markets.
Yes, there is a clear desire for numeric markets from a creator's perspective evidenced by sorting by new. But, sort by trending and see how many show up. We are currently "baiting" creators into creating something that we know will get minimal interaction.
So then the question becomes can we make changes that increase the number of people betting on them. And the answer is no without significant reworking which would be better spent elsewhere.
There's a ton of issues but the two big fundamental problems are:
1. Numeric markets resolve to "prob"/somewhere in the middle of the range. This reduces room for profit, makes the payouts uncalculatable, and removes the fun of the "all-in or nothing" part of predicting with play money. It's almost like playing a game of Storybook Brawl but your choosing a hero at the start of the game that says you can only finish in either 4th or 5th.
2. There is no liquidity. Due to how liquidity is spread across the entire range there is incredibly little liquidity, especially if a large range is set. On most numeric markets I try to bet on I can only bet up to M$10 max.
Anyways in conclusion we should remove them since they mislead market creators and provide a bad experience offering no value. Now there is a second discussion on whether we want to prioritise spending a lot of time making a replacement or not...
@DavidChee Easy solution to stop new users from getting suckered into creating a market noone will bet on while still making experienced users happy:
being able to create a numeric market depends on total investment value being above $5k. Otherwise the button is greyed out with a tooltip that lets them know they can create these when they attain the requisite mana. Win-win as everyone gets the best markets for them and it increases the value of mana!
@MartinRandall I don't think it needs to be for paid accounts, just ones experienced enough or dedicated (via payment) enough to get to 5k
@DavidChee I think both these problems are the same problem - market creators tend to set the ranges too high, and a large flat range leave most of the action at 40-60%. Market creators are led astray by the UI, which uses the terms "MIN" and "MAX" and the hint text "The minimum and maximum numbers". Instead they could use the term "LOW" and "HIGH" and better hint text. This would be a very easy fix and would set expectations better.
@DavidChee The liquidity problem would also be alleviated if we had better ways of providing liquidity. In particular, if it is looking like the answer is going to be between 4.2 and 4.7 or between 42% and 47%, then we should be able to provide extra liquidity in that specific range. And we should be able to charge a fee for providing liquidity and do it without committing all the mana for it up front.
@ian people that have to earn a privilege treat it more carefully as well - if users know that a feature is only for pros and finally get access to it they'll be more likely to read any disclaimer about how to create the market
@MartinRandall in theory that'd be provided by limit orders around that range, yeah? Pretty easy to do right now (though I'm still waiting for a "place a limit order spread" tool so I don't have to enter each separate order).
@MattP One difference is that if I provide AMM liquidity around 40-50% then it stays in play as the probability bounces around. Limit orders disappear when filled and would need to be entered again.
@MattP You can see the difference in a market where there's lots of limit orders. After a few people go wild with big bets all the liquidity vanishes until people start to notice and place new limits. And different liquidity providers end up with different amount of shares once the dust settles, whereas in an AMM many of them would have automatically exited those positions as the probability was corrected.
@DavidChee Partial resolution would also help. Once part of the original range is no longer possible, give bettors a partial payout and redistribute liquidity to the remaining possible range. This allows markets to start out with a larger range and narrow in over time.
Three separate thoughts. First, glancing at the "Newest" list, it looks like numeric markets are maybe ~1/4 newly created markets. This is a clear signal that no, they should not be removed.
Second, I really wish y'all were more comfortable with Manifold being "hard", at least relative to other online things. We have plenty of social networks that are dumbed down, and dumb their users down. Having to learn a few things in order to use Manifold correctly is at least fine. I think it's a very good thing.
Lastly, it seems to me that the clear solution is just to have better onboarding. The first (&second?) time someone opens a numeric market, pull up a little page showing how they work! What the payment mechanism is, and what a good betting strategy is. This is a difficult task in exposition/pedagogy, of course. But again, to me, a great thing about Manifold is that users learn something (if something unconventional, and maybe involuntarily). Lean in!
Now, if you want something technical to change: don't display the max earnings on numeric markets. Even I find that incredibly deceptive. Instead, for a market with range [0,10], you might display a little table of: "if this resolves to 0 you lose M$x. If this resolves to 5 you make M$y." For a few representative values.
Even better might be an interactive graph. The point is to show people the consequences of their bet, when they're making the bet.
To summarize all this long rant: for the love of god don't remove numeric markets. The current implementation is fantastic, it's very useful, quite popular, and if you're worried about people being confused, then there's room to do more teaching.
@ScottLawrence Strong disagree. The current implementation of them is very much not fantastic and has nothing to do with learning. They are actually borderline broken as is.
@SneakySly Of your two concluding complaints, one was that they "mislead market creators" (and this is echoed in the twitter thread and in a user complaint in this thread). That's what I mean by "about learning": a problem for which a reasonable fix is better presentation and pedagogy.
@SneakySly oh, sorry. I'm maybe using words in a funny way.
I think of a UI that makes how things work under the hood clear to the user as "good pedagogy". So to me, a UI change is a presentation/pedagogy change. (As contrasted with getting rid of numeric markets, or changing the underlying market mechanism.)
Don't remove them; we need numerical markets, and without an explicit implementation people will just hack it with mappings to percentage markets.
But maybe retool them to be more intuitive? Up to now I've mostly avoided betting in them as I don't know how they work.
We are also thinking about implementing futures contracts on Manifold, which would solve much (all?) of this use case, and potentially be more intuitive for users.
