See https://www.lesswrong.com/posts/KTEciTeFwL2tTujZk/lw-petrov-day-2022-monday-9-26 .
Edit: You're all so confident so quickly, here's another market to predict exactly when it happens.
1,000
3.00@Multicore It's obviously only pressed because of a bug where they accidentally made a button that takes down the home page and gave everyone the launch codes. Should resolve no.
@NathanpmYoung For folks who bought YES for incentive rather than prediction reasons, I'm curious what they think the true probability was. I saw some big limit orders at 90-95% YES and I mostly didn't trade into them because they felt too low.
@citrinitas What do you mean by "subvert" in "The nice part about prediction markets is that it's hard to subvert them to manipulate external goals."? Prediction markets always incentivize the outcome to move in whichever direction can be influenced in a less costly way. Insofar as destruction is easier than creation, prediction markets are a destructive force.
@MartinRandall It should be possible to play incentives in a prediction market separately from playing probability. Insofar as Manifold is set up correctly, you should always be buying at no more than your true probability / selling at no less than your true probability. The choice of whether to leave (public) limit orders on one side or the other is the way you incentivize action.
@JasonGross I think your question should maybe be directed to @galen ? I was not betting on this market to incentivize action. The vast majority of participants don't even have Manifold accounts.
I guess I should refine what I said above: prediction markets aren't themselves destructive, but being a market maker in a prediction market is (insofar as destruction is easier than creation).
@MartinRandall I was attempting to suggest a partial answer to your question, but on rereading it seems I may have misinterpreted you; you were wondering whether or not the limit orders you saw were there for incentive reasons?
@JasonGross I am pretty sure they were. @galen was buying YES at 95% based on a true belief of 80%. And @NathanpmYoung at the top of the thread is saying that everyone should buy and hold yes with the intention of making a loss here but a win in terms of keeping the site up.
Personally I think the market was pretty accurate, which is why I didn't fill the large YES limit orders that were available. A 15% return in 48 hours would have been a good day.
While pressing the button is an easy action, not creating the button would have been even easier. Manifold users were offering mana incentives to not create a destructive button, but alas they were not large enough.
@Multicore registering my disagreement; the big red button was used and the site was taken down accordingly.
@ZachSteinPerlman The LW team themselves clearly do not consider the button to have been "pushed", given that the site is up.
It seems like the question here is "will the BRB be used [per the stated rules of the event]". Yes, the second part was not actually stated, but it seems a reasonable implication that this market was about user behavior, not programming glitches.
(There's still a good chance this will be a moot point, anyway)
@Retsam19 They don't consider the site to have been blown up. But the button was pressed and launch codes entered.
Aside: When Chris was social engineered in 2020 that counted but when a coder introduced a technical vulnerability in 2022 that doesn't count. Accountability for thee, not for me?
@MartinRandall Using social engineering to convince someone to press the button was within the stated rules of the event in 2020. Someone with zero Karma being able to launch the missiles due to a programming mistake is not within the stated rules of the event.
@Retsam19 A coder causing mass destruction by rushing flawed code into production does seem in keeping with the spirit of the event though.
@Retsam19 I see no references to social engineering in the Petrov Day 2020 announcement. What are you referring to?
https://www.lesswrong.com/posts/XfHXQPPKNY8BXkn72/honoring-petrov-day-on-lesswrong-in-2020
@MartinRandall It wasn't prohibited by the rules, therefore it was allowed. Whereas the rules for this event clearly stated that there's a minimum, (and decreasing) karma requirement - someone with zero karma being able to launch the nukes (at start) violated the rules of the game, so they fixed it.
Obviously, it's a moot point now, but I think the market was absolutely correct to not resolve on the implementation glitch.
@Retsam19 The 2022 post says:
Every user with an existing LessWrong account (created before 2022-09-21) and non-negative karma is able to participate.
Obviously zero is not negative, and so users with non-negative karma were meant to be able to participate. One such user chose to participate.
At the start of the 24-hour celebration, only those with 2,300 karma or more will be able to successfully launch.
This is not a rule, this is a (false) description of how the button works. A rule would instead say something like "You are not allowed to press the button unless you have enough karma to meet the threshold".
It's a moot point for this market, but still relevant for LessWrong users who want to know if they will be treated fairly if they make mistakes, whether coding mistakes or phishing mistakes.
@MartinRandall I don't care to have a semantic debate over "rules" vs however you'd rather describe it.
I don't understand how this is a fairness issue, either. There was a minor technical hiccup which was corrected.
@Retsam19 I'm not sure what your point is about "stated rules", so I don't know how I would describe it.
If you re-read the 2020 postmortem, I think you will agree that Chris's mistake was not treated as a minor technical hiccup. Also, it was not corrected. I might say that it was not a minor technical hiccup precisely because it was not corrected.
Allowing zero karma users to press the button was a worse mistake in some ways. It was made by someone in their professional capacity, in a task that they had opted in to do. If t had not been reversed it would have had a bigger impact, on more users.
These mistakes were treated differently.
@MartinRandall I think there's a valid point about the rules (here I'm simply using rules to refer to LW's stated description of how the Petrov day event would work). In 2020, the site was nuked according to the rules - a user entered their valid launch codes. In 2022, the site was not nuked according to the rules (user entered their launch codes which were supposed to be invalid), but it was nuked according to the code. Those were different due to a software bug.
People certainly can and often should undo problems caused by bugs (Manifold does this all the time - Manifold doesn't use the crypto style "code is law", it follows the intended rules even when the code differs).
@jack I pointed out:
When Chris was social engineered in 2020 that counted but when a coder introduced a technical vulnerability in 2022 that doesn't count.
If I taboo "rules", you are saying "software bug" and I am saying "technical vulnerability" and "coder error" but these are just different words to describe the same thing. This doesn't actually justify a difference in attitude.
People certainly can and often should undo problems caused by bugs.
People certainly can and often should undo problems caused by human error as well, especially if the human error is due to third party deception.
In many games, deception is implicitly against the rules, and the goal is to have fun. You won't get any kudos if you win Connect 4 by convincing your opponent that the goal is to connect them in a square.
Unfortunately LessWrong can't make up its mind whether Petrov Day is a Very Serious Ritual or a Happy Fun Game, and errors are treated inconsistently as a result.
@MartinRandall Yeah, I understand your point, and I think they are both reasonable viewpoints
Thinking about the "how to make systems secure" perspective, yes, technical vulnerabilities and social engineering vulnerabilities are similar.
Thinking about the social game though (using the term game in the sense of game theory), LW designed a game with certain allowed actions and outcomes, and since talking to other people to persuade them to do one thing or another was certainly allowed, I believe the 2020 social engineering is not fundamentally different and was also part of the game (and LW agreed). (Some games disallow deception, clearly this one did not as per LW's statements and actions.)
In the 2022 social game, the "false alarm" is a nonevent - I think it's very much like how the entire LW site briefly going down with 500 errors certainly counted as the site technically going down (DDOS is a real vulnerability) but certainly did not count as the nukes being launched in the social game. I think they are both valid perspectives, and I agree with LW's decision to reverse the bug (because it was not part of the social game) while not reversing the social engineering.
@jack I agree that both views are valid. Obviously I prefer mine.
I am glad that LessWrong has endorsed the Shapely attribution for these continuing failures, which assigns them most of the blame.