@dreev Thanks for the comments.

3. I think self-resolving markets have a weak, roughly constant pull towards truth, that may be countered by a pull towards profit incentives that is proportional to the size of the investments. As people bet larger and larger amounts in this market, the pull towards profit completely overwhelmed the pull towards the stated question (as I would have expected, although I didn't expect the amounts traded to get this large). My initial prediction was that this would end up around 90%, but then the amounts of mana completely blew up and it basically boiled down to what was the ratio of funds between yes and no. I had a spreadsheet where I was calculating equilibrium price = yes / (yes + no) funds for different possible scenarios.

4. I still don't think quiescence alone works robustly, and I think it looks pretty similar to a long Poisson process. 3 expected days is a relatively short Poisson process and I think a 30 expected day Poisson process (for example) would incentivize playing for the long game more. But I think it still plays out like a game either way, and same with quiescence - the price is fundamentally determined by the ratio of funds.

5. Yeah, agreed that the last-24-hours method was not ideal. But I don't think the impact was necessarily that huge. Team no was playing the same strategy as we would have with a close-to-random-timestamp rule or anything like that. Team no strategy was driven by, if we drive the price down by x% for y minutes, how much expected profit is that? That's why we were willing to buy NO down to 10%, even though we were sure that it would resolve much higher than that - because the vast majority of the profit was from the shares already held, the trading during the last day was a small fraction of that.

Another point is that I think if you ran the same market 100 times, you'd get a lot of very different outcomes depending on who decided to participate. And I think this is true of these experiments in general - so I would not rely on a single or even a few experiments to tell whether one rule set plays out differently than another. My prediction going in to this was

self-resolving markets are probably accurate within about plus or minus 20% most of the time, with very large, blatant discrepancies <10% of the time.

and I think we ended up in one of the <10% cases here.

Actually I made a mistake in the average script (again) - if I got it right this time the average was 50.6%, not 69% (which was the result when I erroneously averaged everything from 9/25 and 9/26, not just 9/26).

Here's my script if people want to use it: https://gist.github.com/jackc271/5f85902200445df745eccbab7240cc1f

I want to say, I had a lot of fun making and trading in this market and I think we learned a lot of fascinating things. I'm glad that at least I was paid out in popcorn points :)

@jack Let's all share our numbers and dream about what might be.

Invested M$4,046

Profit M$36,990

+239.4%

I have literally no idea what that percentage means.

@MartinRandall :) Note that this is the hypothetical profit if it resolved at 50%. I think the average over the last day is at about 69%, if the coin flip had been <1/3.

Here's mine:

Invested

M$12,701

Profit

M$97,691 +512.6%

I added some more discussion on the concerns with the current Poisson process (using an average of the previous 24 hours on close) in the comment thread below. And:

I believe that simply closing at a random timestamp and resolving to MKT then works fine and has no such fairness problem. It does still have the "needing to sleep" problem - i.e. potentially advantaging people who can trade more hours of the day, and incenting people not to sleep. I think this usually isn't a big deal, but it becomes one when the amount of mana at stake becomes this large. As mentioned above, some plausible solutions to this could involve limiting trading hours somehow, but I think that creates more problems than it solves.

Also, it's very interesting to note that this market functioned worse and worse as larger amounts of mana were at stake. I think this is generally true of resolve-to-mkt in my experience, and it makes sense because I would say there's a moderate tendency towards truth typically, but a large enough profit incentive can easily override that. In this market, as NO bettors built up larger positions, I went from simply predicting that 99% was overpriced given the market structure, to actively being incentivized to "win" the market for profits.

And obviously this market did terribly at answering the question, whereas if this market had not been [resolves to MKT] it would have resolved YES right away on 9/15. But I still don't think resolve-to-mkt is a clearly bad idea for ambiguous, hard-to-resolve questions in general. What I think is absolutely clear is that for markets that get more trading interest, it's important to have some guardrails. Simple ones can go a long way, like author discretion to resolve if the community comes to a consensus, or if the market probability is obviously wrong. I think you still have to be aware that it isn't robust, but it can work ok on average. And the main other option is polls, which work sometimes, but polls aren't as good at updating on information as markets, for example.

Using this self-resolving market as an example, I think it's providing some types information that would be much harder to get from a poll, such as aggregating new information from Magnus Carlsen's statement. If you ran a poll, you'd probably have to first decide that it's an event worth updating on, and then go out and poll a bunch of people again to get their thoughts. Here, a much smaller number of traders can update the market probability much more efficiently.

@jack Thanks for creating this market! I, too, had a really good time trading along -- I got a lot more invested (mana and emotionally) than expected :sweat:

I think self-resolving might still be pretty good if there was an eg 10% chance of a genuine resolution? This is interesting to me because leverage on different markets would allow for more market estimations with lower resolution costs...

@NinthCause :)

Yeah, that's an idea I've thought about as well! I think it helps substantially, but on a market like this one, 10% chance of me resolving it without self-resolution would still have left massive price incentive to manipulate the market, so I still think it would have failed. But it definitely changes the psychology a lot, which is sometimes even more important - empirically having some small chance of external resolution appears to make an outsize difference.

After analyzing the effects of the bug more closely, I believe it substantially changed the course of the trading on this market. This is the main reason I plan to resolve N/A (I'll leave 24 hours from when I closed last night, just in case).

The way this market turned out is not the way it would have without the bug, although it still is a way it very well could have gone, so it is still good evidence about the fact that resolve-to-mkt can fail badly, especially when the mana stakes are very high.

If it had just been the disagreements about the Poisson process, I think NinthCause was satisfied with them beforehand, and dreev was not. Since NinthCause was by far the biggest YES buyer, we probably could have worked something out where we exchanged shares to essentially undo dreev's position. But since it was not just the Poisson process disagreement, it'll have to be N/A.

Correction: I meant 9/26 to 9/30.

Current proposal: Starting 9/26, shortly after the end of each day (UTC time), close with probability 1/3 using a public verifiable random number generator. After the end of September 30, close unconditionally.

If the market closed after the end of the day, then compute an (exact or approximate) average (mean) market probability over that preceding day (24 hours, midnight to midnight UTC time), and resolve to this value.

The averaging will be computed as follows:

- If someone has an API script that can compute it exactly, we'll use that. The mean market probability over time is simply the sum over each pair of consecutive bets of the market probability times the length of time, divided by 24 hours. (Haven't been able to get it to work myself as of yet, pymanifold is giving me some errors when I try to retrieve market data.)

- If not, I'll use public RNG to generate 10 random timestamps, and compute the mean of the market probabilities at those 10 timestamps.

To walk through this with an example: The first RNG check will be after the end of 9/26, i.e. around 9/27 12:01 am UTC. We generate a random real x between 0 and 1 via public RNG, and check if x < 1/3. If it is, then we close the market, compute the (exact or approximate) mean market probability over the day 9/26 (24 hours midnight to midnight UTC time), and resolve to that. If x >= 1/3, we repeat the procedure the next day.

Note: midnight UTC = 8pm Eastern, 5pm Pacific.

This plan will be finalized before midnight UTC today (9/25) if there are no major revisions based on community feedback.

Ok, no further comments or objections were raised, this is the randomization procedure we are going with. Thanks for all of the input!

@jack Is it too late to voice objections? I don't think there should ever be a hard cutoff. That was the main thing we wanted to avoid, sniping. I guess I also don't like the averaging scheme. There are bots pushing the price down and I have to, y'know, sleep.

The more I think about it the more the average-price resolution feels unfair. It's at odds with how we've been reasoning about it all along and we don't have a good way to compute that average price. I mean, we could replicate the script but that feels like an unfair burden -- something we needed to have known before getting invested in this.

I don't see a problem with cutoffs if you do the averaging. Averaging on it's own should be enough to prevent sniping.
On the other hand I agree that average price can be very confusing. I don't think anyone expected the price to be so volatile.

@Yev Exactly. The averaging solves the cutoff problem but introduces another problem that I'm not sure is less bad.

PS: I think it's still within 24 hours of that proposal? Regardless, I don't think it should be too late to object! Being resistant to price manipulation bots was a key stipulation of the whole experiment.

@dreev I think it's too late to change today's resolution criteria, but we can probably do something different tomorrow if the 3-sided coin doesn't land heads.

@Yev That doesn't seem right to me. I think we have a good argument for why the averaging scheme isn't fair after all, so we shouldn't do that. I guess people have been betting furiously under the assumption that that was finalized? What do we do if there's just nothing that can be done that's fair to everyone? Resolve N/A?

(Btw, if this does have to be resolved N/A then, to be clear, I'll have been super maximally wrong in my original claim that auto-resolving is reasonably robust to price-manipulation shenanigans.)

@dreev People have already spend thousands of mana trying to keep the price low during this averaging window. It'd be unfair to them to suddenly declare that none of it mattered.

N/A would be fair (although quite anticlimactic).

Oof, yeah. But to be clear, if the averaging scheme stands I'm crying foul, for the reasons I've articulated above, plus just the fact that it's asking too much to follow the discussion that closely, as I failed to do till now and till after I'd spent a ton of mana betting under the previous assumptions. (Maybe there should be some mechanism to alert traders to a change in resolution criteria? And probably 24 hours is a lower bound on a fair amount of warning?)

It would certainly be anticlimactic to resolve N/A. What about pure quiescence where the market can just stay open indefinitely? Probably eventually it would quiesce? The sky-is-blue market did. Arguably that's still unfair to the people who thought they just needed to keep the price briefly suppressed?

But if so, that just drives home the point that the averaging scheme fails at the key stipulation about price manipulation resistance.

PS: This has me wondering -- and I don't want to sound like I'm trying to salvage my original claim about auto-resolving markets cuz I'm totally ready to eat humble pie there -- why not just always use pure quiescence? Some arguments:

1. The loans feature means people's mana isn't tied up indefinitely.

2. The market still may be (in general, not in this market) providing useful information even as it oscillates.

3. We've talked about how auto-resolving fares better if there's even a chance of ground-truth resolution. Resolving only to what the market quiesces to is sorta similar to that.

4. It becomes kind of a conditional market: what will we agree the market price should be, if we do agree?

In short, an auto-resolving market is like saying "this resolves to whatever we all agree it should resolve to". Until it quiesces, we haven't all agreed yet, so it can't resolve yet. Maybe that's ok-ish?

PPS: As I'm typing this I'm trying to respond to the bots and push the price back up but I have no idea how to reason about the new rules and it feels like a totally new game we're playing. So I just want to reiterate -- and still within 24 hours! -- my strong objection to the new resolution rules.

@dreev Thanks for bringing up your concerns, it's never too late to voice them, although I'll have to think carefully about any changes to ensure fairness. I think at least it is probably possible to change the resolution procedure for 9/27 and onward without being too unfair to anyone. I thought you were happy with the broad strokes of the proposal (including the fact that it would run 24/7 in particular) based on your earlier comments, I'm sorry if I misunderstood. In retrospect I should have explicitly polled the major participants to get a consensus on the final proposal, but I felt it had not really undergone any substantial changes from the earlier proposals so we had agreement already.

Firstly, I want to reiterate that I am committed to ensuring that the resolution process is a fair representation of the market probability. I tried to come up with a mechanism to be as fair as possible to all participants, and I will continue to make adjustments if they can improve the fairness.

1. Average-price resolution

> It's at odds with how we've been reasoning about it all along and we don't have a good way to compute that average price. I mean, we could replicate the script but that feels like an unfair burden -- something we needed to have known before getting invested in this.

The averaging is simply modifying a previous proposal, which was to randomly resolve at any point in time. That's easy and fair but highly volatile. I believe averaging preserves roughly the same expected resolution, while reducing the volatility when prices are wildly oscillating.

Note that the original idea of resolving randomly at any point in time would have exactly the same result: other people trading while you are asleep could move the price, and it could have randomly resolved in the middle of your night. I'll address this point further below.

Regarding the calculation - I got most of a script for it mostly done (https://gist.github.com/jackc271/5f85902200445df745eccbab7240cc1f). And earlier I proposed that if I couldn't get the exact average computation working, then we'd just approximate the average by taking a random sample of 10 timestamps - that would have exactly the same expected value, but with some more noise. I think they are equally fair, it's just some tradeoffs between implementation complexity and noise.

2. Bots and sleep

> I guess I also don't like the averaging scheme. There are bots pushing the price down and I have to, y'know, sleep.

Thinking about this more, I think the issue isn't about averaging, it's about the fact that a) the market is open for trading 24/7 and b) the market price at all hours is being counted in the resolution mechanism. Would you agree with that assessment?

Note that I had always assumed in the discussion of the Poisson process that it would effectively run 24/7 unless we explicitly chose not to - see Martin and my comments in the discussion threads on both markets. (On my lower-stakes markets, I typically just say I'll resolve at some arbitrary time in the day, but I want to avoid any potential bias on higher-stakes markets like this one.)

A hypothetical: let's say we kept the market open only during some chosen market hours during the day. (Doesn't have to be stock market hours, could be whatever is most convenient for the traders). This has the advantage of making it easier on the traders, and concentrating liquidity during that period of trading. It has the disadvantage that people are on different timezones and different schedules, and there are e.g. UK traders who would be trading at very different hours than US traders. Leaving the market open 24/7 makes more fair between participants in different timezones.

Similarly, we could hypothetically use market data only from specified hours of the day for resolution. E.g. we could say that the market will only resolve based on the market price during the daytime PDT, or only between 7pm and 9pm, or whatever. (This could be implemented as only having the Poisson active during those hours, or only using those hours for the average, it works out basically the same.)

It does have some advantages, if all the major market participants can agree on some good hours, I am happy to use those, but I don't want to unfairly disadvantage someone in a different timezone.

3. Hard cutoff

> I don't think there should ever be a hard cutoff. That was the main thing we wanted to avoid, sniping.

I don't think it's an issue. Even when we hit 9/30, we are still resolving to a random/average price during the day, so it still avoids the last-minute-trading problem. One of my previous proposals was to not even have the 5-day window but instead just a 1-day window, I think that would still work fine.

And the reason I want it is I think it is it's a big help for traders to know they'll get their payouts by some predictable date, and it's a big help for me as the author who has to keep operating the procedure every day until it's closed. That said, I'm not opposed to removing or adjusting the hard cutoff, if that's what people prefer. For example, another option is to gradually increase the probability of close every day e.g. increase it by 10% every day.

4. Why not just quiescence?

Because the market might never settle, or take absurdly long to do so. I would bet it would be months at least. An author could certainly create a market like that, but having a reasonable timeframe for close was always my intent.

Just as an analogy, the stock market is constantly fluctuating, but people still use average prices and daily close prices as meaningful numbers. (A tangential interesting fact: stock close price is far harder to manipulate than a Manifold market, and also illegal to manipulate, but even so there are many stories of people manipulating it! See https://www.bloomberg.com/news/articles/2022-02-16/what-block-trades-are-and-why-the-sec-s-investigating-quicktake for example.)

> The loans feature means people's mana isn't tied up indefinitely.

Loans do not solve the problem of tying up funds. If this market lasted a month or a year, most of my investment funds here would be tied up for most of the duration.

> The market still may be (in general, not in this market) providing useful information even as it oscillates.

Yeah, the timeframe in which the market is open and in which it resolves is a design decision for the market, and different markets call for different lengths of time. E.g. if it is resolving uncertainty about the answer, that's probably good, and could take some time.

> We've talked about how auto-resolving fares better if there's even a chance of ground-truth resolution. Resolving only to what the market quiesces to is sorta similar to that.

Hmm... I don't think that's similar, the whales in this market could also achieve quiescence by putting up massive limit orders around the equilibrium price point.

> It becomes kind of a conditional market: what will we agree the market price should be, if we do agree?

There may be some truth to that, but isn't the market price supposed to be that too? Obviously market price can be manipulated, but so can quiescence I think.

To summarize things a bit, I believe that I can boil down @dreev's main concern here is on averaging vs resolving at a randomized time. I view these as basically equivalent in terms of the expected value of the resolution, and therefore they have almost the same market incentives. Averaging just reduces unnecessary randomness and volatility. But if participants prefer, I am happy to just resolve to a random point in time (that's why I gave both as proposals earlier).

@jack I'd prefer non-random as it reduces variance and (given a public script) is easier to replicate.

@jack I'm opposed to changing the resolution criteria tomorrow, if that becomes relevant, as my bets in this market were based on my expectation on how it would resolve over the five day period. Sorry to be awkward.

Please let me know if I have misunderstood your main concern @dreev, or your other concerns.

A few other comments:

But if so, that just drives home the point that the averaging scheme fails at the key stipulation about price manipulation resistance.

Not at all, my argument all along has been that resolve-to-mkt is fundamentally subject to price manipulation. All this stuff about randomized or averaged resolution is purely to address the smaller problem of last-minute price manipulation. Price manipulation in general is (I claim) impossible to robustly address without some sort of rule that at least potentially allows non-mkt resolution. But we can at least ensure that the market price over a longer period of time matters, to give everyone a chance to trade and have price input on the market resolution.

(Maybe there should be some mechanism to alert traders to a change in resolution criteria? And probably 24 hours is a lower bound on a fair amount of warning?)

Copying my earlier comment: I thought you were happy with the broad strokes of the proposal based on your earlier comments, I'm sorry if I misunderstood. In retrospect I should have explicitly polled the major participants to get a consensus on the final proposal, but I felt it had not really undergone any substantial changes from the earlier proposals so we had agreement already.

To add some more details: I posted the first outline https://manifold.markets/jack/will-biden-be-president-on-915-reso#Kw05SzzVLw7jKREuugpM 9/23, so there were two full days (9/24 and 9/25) for comments. This outline has what I think is the key point, which is using a Poisson process to close at the end of a random day, and generating a random timestamp during that preceding day and resolving to MKT prob at that time. And I interpreted your comments as meaning you were happy with this proposal (please correct me if that was not true).

Then, based on discussion there and in my market on how to resolve-to-mkt, on 9/24 I posted a more detailed set of proposals https://manifold.markets/jack/will-biden-be-president-on-915-reso#heILj7OnjOh9AeyDTN8b which included the proposal of either averaging over the day, or resolving to a random timestamp during the day. My view is all the variant proposals are fair, and I don't have a strong opinion between them. But based on discussion, people were leaning towards preferring averaging.

So on 9/25 morning I picked the averaging proposal with the intent that it would come into effect at midnight UTC. I did this because I didn't want the process to drag on, and in my mind averaging and randomization did not substantially change market incentives, and the comments so far had agreed that they were basically equivalent in terms of market structure but preferred averaging as simpler and lower-variance.

@MartinRandall

I'm opposed to changing the resolution criteria tomorrow, if that becomes relevant, as my bets in this market were based on my expectation on how it would resolve over the five day period. Sorry to be awkward.

Totally fair and understandable! However, I am optimistic that we can either explain why the current system is fair to everyone's satisfaction, or make minor adjustments that everyone agrees are fair as well.

Also I want to be clear that everyone was aware that we were going to add the poisson process if quiescence didn't happen soon enough, and team NO bet accordingly - during the pre-poisson period team NO built up a huge position of NO shares bought at good prices (90s), while ensuring that the price did not quiesce; and then when the poisson window opened they moved to "correct the market mispricing". I don't think the details of the poisson process make a substantial difference, either way this was the team NO plan from the beginning.

(Think of this analogy: if this were a normal author-resolved market on whether Musk would win the lawsuit vs Twitter, and team NO had insider info that they knew would remain private until the trial started, and team NO therefore gradually builds up a large position at favorable prices prior to that date. Then the trial starts and the price starts moving a ton.)

To close the loop on this a bit, I was talking with @dreev on discord and actually the issue wasn't averaging, he agreed he had the same objection to using a random timestamp in the previous 24 hours. After more thought, I'm also now convinced that it does have a small but noticeable fairness problem in that the market price early in the day has an anchoring effect on the rest of the day. Firstly, consider if you don't have any Poisson process and just average over 24 hours - then halfway through the day, half of the price has already been determined and you have upper and lower bounds on the outcome with a range of 50%. With the Poisson process as well giving a 1/3 chance of close, instead of being 1/2 determined it's actually 1/6 determined halfway through the day, which is not nearly as bad but still a substantial amount.

I had briefly considered this issue when thinking about the design, but had not thought it would be nearly as impactful as it seems to have been, and I had also thought it would advantage team YES more than team NO (who all agreed to it), when in fact it seems to have ended up being the reverse.

I believe that simply closing at a random timestamp and resolving to MKT then works fine and has no such fairness problem. It does still have the "needing to sleep" problem - i.e. potentially advantaging people who can trade more hours of the day, and incenting people not to sleep. I think this usually isn't a big deal, but it becomes one when the amount of mana at stake becomes this large. As mentioned above, some plausible solutions to this could involve limiting trading hours somehow, but I think that creates more problems than it solves.

Another possible approach to market design (not for this market, but in general) is to take a more quiescene-oriented approach. One idea I had considered was to start with a threshold of closing the market if the price stayed within 1% for 24 hours, then gradually increase it (say 1% per day). This has some pros and cons, and you'd still need to figure out how to actually pick the resolution value - if you try to use say the midpoint of the range is manipulable, and if you try to use average over the day that has the above problems.

@jack I honestly thought this was a pretty obvious consequence of your mechanism design, and that anyone with several thousand mana on the market would realize it and act (or object) accordingly. Both sides had fair and equal access to the same strategies - recruiting partners, writing bots, setting limit orders, etc. This is why I sold out 10,000 NO shares for at m600 at the start, I thought this was a good price.

I was completely wrong-footed when I later realized that Daniel and Austin weren't paying any attention to the market during the crucial initial stages of the Poisson process, and perhaps weren't playing to win. I was further confused that nobody else bought YES shares at ultra-low prices like 20%, which would have given them a 3-4x return with minimal risk. To repeat something I said before on this market, nobody behaved like I predicted.

@MartinRandall Yeah, it was super interesting. I think NinthCause was a couple hours late (quite possibly because they were busy with building Manifold lol) and then put up a big batch of limit orders, which wasn't an entirely unreasonable strat given they were assuming that their limit orders would hold, or even baiting NO into buying mana to fill them.

Also, for reference, the structure of the Poisson process here is consistent with what I had in mind from the beginning and I think the market text is reasonably clear about it, although @dreev interpreted it very differently. It's also consistent with what I have been doing on my other self-resolving markets (see https://manifold.markets/jack/what-is-the-probability-that-a-coin or https://manifold.markets/jack/how-clear-do-you-think-manifolds-co for example), but just with a more formalized / secure / verifiable way to do the randomization in order to avoid any bias or potential for bias.

@jack I was inspired by:

> "an incentive to try to hold the market probability more steady"

I don't think your proposals so far do that - if the market over the last 24 hours spent 12 hours oscillating rapidly between 40 and 60%, and 12 hours stuck at 75%, it doesn't put extra weight on the 12 hour period of stability. But possibly if it did, that would be better!

So, proposal: every day generate a number N from 1-24. If there is a period of stability of >N hours, resolve to that probability. If multiple, resolve to the longest period. If no sufficiently stable period, try again tomorrow.

@MartinRandall Interesting idea. I think there's some potential benefits, but I'll have to think more carefully about it. The biggest issue that comes to mind is that it would tend to put higher weight on off-peak hours (when most traders are asleep, for example) where the market will be more steady just because it's lower activity. That would then create some unfortunate biases in favor of people who can trade during those times.

@jack Maybe if we allowed 1% variations to still count as "stable" then someone could put large limit orders at 90% and 91% to encourage the market to be stable while they are asleep.

Also, I think in your example earlier, it's not at all clear to me that 75% is a more correct probability than 50% (the midpoint of the oscillations). But I do think it is likely good to incentivize quiescence, and if the mechanism can do that in a fair way, that seems good.

Another idea: Each day, we check the max - min within that day, and if it's less than a threshold d then we close to an average (perhaps randomized) within that day. Then, we increase the threshold d for the next day. For example, it could start at 1% and increase by 1% per day.

My idea doesn't do anything particularly to encourage stability though. Also, an issue with any of these mechanisms is that if someone doesn't like the price, they are incented to move the price a bunch (even temporarily). The good thing is that with limit orders that is likely to be costly.

@jack Another approach is to look at the limit orders. If there is a m1000 limit order wall at 80% and another at 90%, then resolving to 85% seems "fair" to both sides. There'd need to be some actual math behind that, though.

If someone tries to exploit that mechanism by putting a massive limit order in, then they are risking someone taking them up on the limit order, so it's a high risk approach.

The slowly expanding quiescence window is interesting, I considered that for my market. Specifically, I considered letting the probability of close be based on the probability range over the last day.

@jack The first approach doesn't require any randomness. The probability is a piecewise-constant function, it's super easy to average.

@Yev Yeah, if someone hands me a script to do it, I'll happily use that. I had some trouble trying to get API stuff working last time I tried.

I wrote a script to try to compute the average market prob, it mostly works but I don't entirely understand the data model, it is not true that probAfter of one bet always matches probBefore of another bet (probably something related to limit orders is my guess). If I don't have something I am confident will compute a correct average, I will use the manual average of random sample approach.

Ok, I checked that there was only one discrepancy and it was very small, so even with that discrepancy the average computation should still be correct.

@dreev My daily profit numbers have certainly been... exciting these last few days.

@dreev Wait, sorry, what I said was all wrong/confused. I think.

Agreed about our profit numbers being exciting!

@Pepe Haha not really, there are a lot of us with very large NO positions.

I wrote up a variety of my ideas here: https://manifold.markets/jack/what-are-the-best-ways-to-operate-a.

I spent a little time trying to put together something that continuously generates random numbers with drand (a distributed public randomness beacon) and uses the API to close the market with probability p, but I wasn't able to get the drand part working after an hour. Just doing it with local RNG would probably be much easier but would lose the public verifiability. Also I don't currently have an easy way to run such a script continuously. Therefore, I'm currently leaning towards something that is easier for me to run semi-automated and semi-manual.

One idea I was thinking about was like this: once every day, check RNG and close with probability p. If closing, generate random timestamp in the previous day and resolve to the probability at that time. The RNG would be via public randomness beacon so everyone can verify it. I'm especially curious what people think about the idea of retroactively resolving to the market probability at a previous point in time. I think it still works at aligning the incentives properly, even though it might feel weird for bets after that to "not count" (they do count in a probabilistic sense).

Would love to hear your ideas and suggestions and feedback, both on this specific idea and the general brainstorming.

I forgot to mention, my intent is for the expected market close time to be roughly within the period 9/15 to 9/30 (this is similar to some of the other self-resolving markets I've been making like https://manifold.markets/jack/did-hans-niemann-cheat-against-magn).

15 days is already a kind of absurdly long amount of time to determine whether Biden was president :)

@jack Thanks for the excellent research here! A property of Poisson processes that could simplify this is that the gaps between when the process fires (to close the market, in our case) follow an exponential distribution. That means you don't have to repeatedly randomize to know when the Poisson process fires. You can take a single random number and derive the amount of time until it fires. In particular, if you want it to fire in d days in expectation you take a U[0,1] random variable X and compute -d*log(X). That's how many days until the market should close.

@dreev Yes,the easy way is to pre-generate a random timestamp. However, this is cheatable because someone (me) could know the timestamp and trade based on it. I am of course an honest resolver, but I also want the public to be able to verify my honesty, and for other authors to use similar mechanisms that prevent them from cheating.

my intent is for the expected market close time to be roughly within the period 9/15 to 9/30

@jack I can't help but notice that most of that interval is in the past :D

@jack Also, to add to my last comment, I think simple is good, authors (myself included) generally don't want to do something that's way more work to operate. But I'm optimistic that highly secure and verifiable mechanisms can be done without a lot of extra work - e.g. the one I gave seems like not much extra work, just checking a daily RNG source (which can be automated relatively easily).

@jack An interesting way to get a random unpredictable timestamp is to take some publicly-verifiable realtime data, generate a hash, and use that as the random result. For example, some realtime weather or stock data to a SHA hash. Treat that as a value in [0,1], and test if it's lower than some point in a given distribution (such as the earlier-suggested Poisson distribution).

One catch: it would be best if the seed snapshot was "chunky" in time somehow, otherwise you can quibble about exactly when the snapshot was taken, and you also have to continuously compute the hash, and you will miss snapshots while computing the current one.

You could do something like Dow Jones + S&P 500 + other prices at market close, but that only gives you one-day resolution.

@SteveHollasch I already know of several ways to generate public randomness, such as https://drand.love/ - the main question is finding one that is as easy to use as possible. drand would be perfect for my use case if only I could get the client to work!

Bitcoin hashes are another pretty reasonable option here. Realtime weather or stock data that is hashed could be a reasonable option if one chooses a specific data feed and uses all the values with timestamps that are exactly on the minute, for example - but I feel like that would end up being more work and less reliable than other options.

@SneakySly Less accurate AND takes longer to resolve!

@SneakySly Thanks for the heads up. Who let it get all the way back up to 91%?

@MartinRandall Yup! That the market is still being traded in past the date in question makes these markets not good sources of information.

@SneakySly Yep, this market is going pretty much as I expected.

Similar to what Martin wrote in his market, the extensions won't go on forever, my current thinking is after ~1 week to switch to a Poisson like process and for it to close no later than the end of the month (but this can be adjusted if there are better suggestions)

Also, the taking longer to resolve part can certainly be reduced, although potentially at the cost of even worse accuracy. E.g. on some self-resolving markets I've simply set the resolution to be that I'll resolve MKT quasi-randomly in a 2-day period - and that can be adjusted depending on how much you want to protect against last-minute price swings vs quick resolution.

For reference, my belief based on previous experience was that self-resolving markets are probably accurate within about plus or minus 20% most of the time, with very large, blatant discrepancies <10% of the time. So far these experiments are consistent with that.

For a highly subjective or poll-style question that is hard to resolve any other way, this is reasonable. For a question that can be resolved the normal way, it's not very good.

@MartinRandall President is not automatically capitalized so it is the same question. Lower case is grammatically correct.

@BTE I'm learning all kinds of things today.

@MartinRandall That means it’s a good day ;-)

@MartinRandall You accused me of weaseling in the "sky is blue" market and now this? The whole point of this experiment is to see what happens when there's no ambiguity about the true answer. So creating ambiguity seems... dishonorable, I guess, to throw your adjective back at you. I guess your reasoning is that this is the kind of thing that would happen in a resolve-to-MKT world and so it's a fair test of my prediction? I can see that side of the argument so no hard feelings! (Also I may have totally misunderstood your "not as obvious as you might think" comment. I'm impressed with your markets and comments and technical acumen in general so 99% chance my perception of hostility from you in these discussions originates in a misunderstanding.)

@dreev Based on how you responded in the other market I expected you to ask @jack to modify the market question. I'm really doing badly on predicting this set of markets, nobody is behaving how I expected.

@dreev I specifically did not try to make this question bulletproof unambiguous. For example, I never specified "US President", because most reasonable readers would understand "President" as clear in context. I stated a question in plain English which I think has a clear meaning, but it's always possible for the market to figure out something unclear about it.

It does not surprise me in the slightest for people to try to throw weird technicalities at the question to recruit people to their side. It already happened on one of my other recent markets here: https://manifold.markets/jack/will-donald-trump-be-us-president-o#q1O2C18FpFvhTBDJ0GxD.

@MattP This experiment is with a clear-cut question that doesn't need a self-resolving mechanism, but there's been a lot of interest in self-resolving mechanisms for more ambiguous questions, both on Manifold and other prediction platforms. These markets can ask subjective hard-to-resolve questions, and they do often work in practice, there's a lot of past examples that went fine. They also go poorly sometimes, but it's more rare in my experience. Here are just a couple examples, there's a lot of these poll-type markets.

The issue of last minute movement is handled by the clause about extending close after last-minute trades.

Btw I moved that clause up from the bottom to the top of the description since both comments asked about it.

@jack Hmm, that it is. Seems like a lot of work for you, heh - but first glance the mechanism seems fairly sound.

@MattP I don't think it'll be too bad (fingers crossed lol). But in theory this could be automated via the API (or a built-in option on Manifold).

Also in the past I've often just said "I'll resolve at a semi-arbitrary time in this window" and it's worked fine. If there was an option to resolve at a random time in a window, that would be simple and I think essentially prevent last-minute manipulation from being a viable strategy.

@BTE Right, those are the sorts of things Martin and I argued would cause problems with self-resolving markets. I think just how big those problems are likely to be is an unknown though, which is why I'm interested in the results of this experiment.

Waiting until last minute before close is (at least partly) addressed by the clause about extending the close date after trades.