Tommy McBayes is on trial for murder! He’s either innocent or guilty, and you will have to figure out which, with the help of your fellow Manifolders! We earnestly hope that all will cooperatively seek the correct verdict and avoid bayesian persuasion.
YES to convict
NO to acquit
Every 2 days, between 7 PM PST and 9 PM PST, the organizers will see which participant holds the largest number of shares in this market (either YES or NO), and ask that participant in the comments to propose a test.
This test will return “guilty” or “innocent”, but has a certain rate of false negatives, and the selected participant will publicly set these two probabilities.
For example, if they select “60 guilty/50 innocent”, then the test will be accurate 60 percent of the time if they are guilty, and will be accurate 50 percent of the time if they are innocent.
The organizers conduct the test within the next day, and will publicly comment whether the test returned guilty or innocent.
Unfortunately, the court’s most accurate tests take more time to prepare. Initially, tests can have false negatives from 40-60, and with each testing cycle, more accurate tests are possible (35-65 for the second test, 30-70 for the third test, etc.).
Tommy’s trial will run for 20 days/10 tests, or until the market is at/above 99% or at/below 1% at the time of testing.
The first day of testing is January 30. The organizers (@KCS , @Nadja_L) have flipped a coin and have taken note of Tommy’s guilt or innocence. Obviously, the organizers will not trade.
Tests:
(40-60 permitted, @jcb chose 60/60): innocent
(35-65 permitted, @jcb chose 65/65): innocent
(30-70 permitted, @MartinModrak chose 37/61): guilty
(25-75 permitted, @MartinModrak chose 51/73): guilty
(20-80 permitted, @prigoryan chose 20/20): guilty
(15-85 permitted, @yaboi69 chose 85/50): innocent
(10-90 permitted, @MartinModrak chose 90/90): guilty
(5-95 permitted, @prigoryan chose 95/95): guilty
(1-99 permitted, @prigoryan chose 99/99): guilty
(0-100 permitted, @prigoryan chose 50/1): innocent
Close date updated to 2023-02-19 11:59 pm
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ2,758 | |
| 2 | Ṁ87 | |
| 3 | Ṁ28 | |
| 4 | Ṁ27 | |
| 5 | Ṁ27 |
So yeah, Tommy was in fact guilty, as decided by an initial coin flip. @Nadja_L and I really enjoyed organizing this, and learned a lot, including that this market structure doesn't reward bayesian persuasion after all! Thank you everyone for participating, and our condolences that the RNG really screwed some people over. More final thoughts:
@KCS ^ This was the code used for RNG. If random() was greater than the proposed test accuracy, the test accurately returned guilty. If you sha256 the above comment (without a trailing line break), you'll see that it matches the certificate of authenticity posted earlier: 594d04906dd35726bc92949295d1352521e801e9bfc514509659d08d62e54154
@prigoryan You have the most shares again! For test #10, please specify what you would like the accuracy-given-Tom-is-guilty and the accuracy-given-Tom-is-innocent to be. The two probabilities must be integers between 0 and 100.
@prigoryan The test returned innocent! The whole court room was a little disappointed that the Omniscient Oracle would not be called to testify after all.
@prigoryan You have the most shares again! For test #9, please specify what you would like the accuracy-given-Tom-is-guilty and the accuracy-given-Tom-is-innocent to be. The two probabilities must be integers between 1 and 99.
@prigoryan The test returned guilty! The jury has turned on Tommy, and no longer seem amused by his proclamations of innocence. For Tommy's part, he has drastically reduced his interactions with the press.
@prigoryan You have the most shares again! For test #8, please specify what you would like the accuracy-given-Tom-is-guilty and the accuracy-given-Tom-is-innocent to be. The two probabilities must be integers between 5 and 95.
@prigoryan The test returned guilty! Tommy was visibly distressed as the prosecution seized onto yet another tasty morsel of evidence. Sweat beads could be seen forming on his face.
@MartinModrak You have the most shares again! For test #7, please specify what you would like the accuracy-given-Tom-is-guilty and the accuracy-given-Tom-is-innocent to be. The two probabilities must be integers between 10 and 90.
@MartinModrak The test returned guilty! Tommy's counsel appeared unruffled by the result, but the same could not be said for Tommy himself.
@yaboi69 You have the most shares! For test #6, please specify what you would like the accuracy-given-Tom-is guilty and the accuracy-given-Tom is innocent to be. The two probabilities must be integers between 15 and 85.
@Nadja_L Let’s go with 85% accurate if guilty, and 50% accurate if innocent, please.
Sharing some thoughts to check my understanding:
Bayesian persuasion happens when TP and TN rates are different (one group is misclassified more often). In the paper’s example, all guilty and only about half of the innocents are classified correctly, which gives the prosecutor a higher success rate.
How much you can bias the results is of course limited because the parameters are public. Going under 50% is not helpful for this specific effect, because a test with a preset 15% true positive rate is equivalent to an 85% one, just with the results flipped. It may, however, mislead people who only look at the labels.
The paper presents a way for the prosecutor to figure out the optimal parameters, but I didn’t parse it. One thing I don’t understand is why is the optimum TN 4/7 in their example. Maybe 7 makes sense because 7/10 is the prior to cancel out, but where does the 4 come from? Why is it not 35/70, ie. a completely uninformative 50-50 test?
Since the market resolves to the truth, there’s less of an incentive to mislead others (because we’re getting just as misled). If it resolved to the judge’s decision, it would be worse though, degrading the market into a fancy lottery. A mixed form could be interesting.
@yaboi69 The test returned innocent! Tommy held a press conference further proclaiming his innocence, stating while he looked forward to being absolved by the jurors that this entire affair has been ludicrous. The prosecution seemed defeated after the results returned, but vowed to keep fighting.
@yaboi69 The central intuition to understand these results is that when a defendant is more likely to innocent than guilty, the prosecution can design a test which will make it appear that the defendant is just as likely to be guilty as innocent. Because the judge can only decide to convict or not to convict, and prefers to make the correct choice, she will choose whichever is more likely (although there is commonly a tie-breaking rule toward guilt for the technical purpose of ensuring the existence of an equilibrium.)
Therefore it is imperative that upon receiving a test result X which should make the judge convict, the probability that the person is guilty conditional on X should be weakly greater than 50%. To maximize the chance of getting an X result, you want to raise the TP rate to 100%. Then you want to ensure that the total probability of getting a FP is equal to the total probability of getting a TP. In the case specified in the paper, because the probability the defendant is guilty is 30% and the probability they are innocent is 70%, the FP rate must be equal to 3/7 (implying the TP rate is 4/7.) Therefore, the prosecutor can convince the judge to convict 60% of the time, despite the fact that she knows the defendant is guilty at most 30% of the time.
One other note: while the fact that the parameters are public does limit how much results can be biased, this limit is necessary to work. Without the extreme commitment power to run an experiment which the judge knows the parameters of, the game just devolves into cheap talk. Any weakening of the commitment power actually makes the prosecutor worse off.
@yaboi69 One should also keep in mind that Bayesian persuasion works only for some types of decisions/utility functions. For the context of a binary guilty-innocent decision, I can benefit from persuasion only if plotting my expected payoff against guilty probability of the judge would show non-convex regions. So if both I and other traders value M$ linearly (e.g. I value an increase from M$ 10 to M$ 20 the same as increase from M$ 100 to M$ 110) there is no way for me to gain from persuasion. If other traders valued M$ linearly, but I was willing to sacrifice small but quite sure gains for larger but substantially less sure gains, then (if I understand the paper correctly) it would be sensible for me to choose tests that are just somewhat reliable so that other traders are likely to update on them but not too reliable, so there is non-negligible chance of them giving the incorrect answer. I would then bet as if the test was incorrect. So on average I would lose money, but in the case the test gave incorrect answer, I'd get a large payoff and since I value uncertain large payoff more than reliable small payoff, I am better off.
Hope that makes sense and that I understood the paper correctly.
@prigoryan You have the most shares! For test #5, please specify what you would like the accuracy-given-Tom-is-guilty, and the accuracy-given-Tom-is-innocent. The two probabilities must be integers between 20 and 80.
@MartinModrak You have the most shares once again! For test #4, please specify what you would like the accuracy-given-Tom-is-guilty, and the accuracy-given-Tom-is-innocent. The two probabilities must be integers between 25 and 75.