Specifically, this is about predicting which states end up voting for which presidential candidate.
(Default 538 model.)
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ1,150 | |
| 2 | Ṁ367 | |
| 3 | Ṁ200 | |
| 4 | Ṁ160 | |
| 5 | Ṁ139 |
People are also trading
As Bolton noticed, my 538 probability for Arizona was wrong; I suspect I looked at Arkansas instead since it's "AR" on the map. I've double-checked all the other 538 numbers and they're all correct. This changes 538's score to 0.03026254, making it a win for Manifold instead, apologies everyone. @mods please re-resolve.
@IsaacKing Am I wrong in saying that the Arizona entry for 538 is wrong? By my calculations this would make it a win for Manifold.
Um, ok, I've just discovered that Manifold's UI shows wildly incorrect probabilities when I zoom in on a specific date.
https://discord.com/channels/915138780216823849/1370569382949818438
So these results may be wrong. Will investigate.
I guess there's an argument that if Manifold is displaying inaccurate probabilities, that is indeed a severe problem with their forecasting accuracy, and they should lose this market because of it. :P
But given that the graph shows the a different probability at a different zoom level, this is IMO obviously just a bug that should be corrected for.
@IsaacKing if you're using the bets then you'll have the correct probs, I made some changes to the graphs recently to cache past points but there is a bug if you zoom in on a section of a MC question where there are no points (bets) in that timeframe
Out of curiosity I ran it with probabilities capped to 0.01/0.99 for 538 (Since Manifold's UI can't go outside those bounds but 538's can) and it has negligible effect. Even capping 538 to go no lower than 10% and no higher than 90% and it still wins. (~0.036.) Manifold got totally destroyed here.
Sorry for the delay. I had said I would use results from 1 week before the election, but there's no 538 archive of that day on the wayback machine, so I'll use 6 days before instead. Here's the 538 forecast:
And here's the Manifold one: https://manifold.markets/election
I'll use Manifold's probability at 22:06:07 Pacific time on October 30th, to be consistent with 538.
And I'll use the displayed probability in the UI for Manifold as well, since that's what people think of as the "Manifold prediction", and it seems unfair to 538 if I were to use an API for Manifold and not for them.
I won't include DC since I had just said "states" in the description, though that was a misthought on my part, I just forgot it existed. Oops.
Chance of Harris victory from each platform. I transcribed these by hand so maybe double-check my work.
538:
Alabama: 0.001
Alaska: 0.03
Arizona: 0.001
Arkansas: 0.001
California: 0.999
Colorado: 0.98
Connecticut: 0.999
Delaware: 0.999
Florida: 0.1
Georgia: 0.36
Hawaii: 0.999
Idaho: 0.001
Illinois: 0.99
Indiana: 0.001
Iowa: 0.06
Kansas: 0.001
Kentucky: 0.001
Louisiana: 0.001
Maine: 0.94
Maryland: 0.999
Massachusetts: 0.999
Michigan: 0.57
Minnesota: 0.87
Mississippi: 0.001
Missouri: 0.01
Montana: 0.001
Nebraska: 0.001
Nevada: 0.47
New Hampshire: 0.85
New Jersey: 0.99
New Mexico: 0.92
New York: 0.999
North Carolina: 0.37
North Dakota: 0.001
Ohio: 0.04
Oklahoma: 0.001
Oregon: 0.99
Pennsylvania: 0.47
Rhode Island: 0.999
South Carolina: 0.01
South Dakota: 0.001
Tennessee: 0.001
Texas: 0.07
Utah: 0.001
Vermont: 0.999
Virginia: 0.9
Washington: 0.999
West Virginia: 0.001
Wisconsin: 0.55
Wyoming: 0.001
Manifold:
Alabama: 0.01
Alaska: 0.08
Arizona: 0.28
Arkansas: 0.02
California: 0.99
Colorado: 0.97
Connecticut: 0.98
Delaware: 0.94
Florida: 0.07
Georgia: 0.32
Hawaii: 0.99
Idaho: 0.02
Illinois: 0.98
Indiana: 0.02
Iowa: 0.06
Kansas: 0.04
Kentucky: 0.33
Louisiana: 0.01
Maine: 0.92
Maryland: 0.97
Massachusetts: 0.9
Michigan: 0.58
Minnesota: 0.91
Mississippi: 0.02
Missouri: 0.02
Montana: 0.02
Nebraska: 0.02
Nevada: 0.38
New Hampshire: 0.85
New Jersey: 0.97
New Mexico: 0.93
New York: 0.98
North Carolina: 0.31
North Dakota: 0.33
Ohio: 0.11
Oklahoma: 0.02
Oregon: 0.33
Pennsylvania: 0.51
Rhode Island: 0.98
South Carolina: 0.03
South Dakota: 0.02
Tennessee: 0.02
Texas: 0.06
Utah: 0.02
Vermont: 0.98
Virginia: 0.88
Washington: 0.99
West Virginia: 0.02
Wisconsin: 0.54
Wyoming: 0.01
Results:
Alabama: 0
Alaska: 0
Arizona: 0
Arkansas: 0
California: 1
Colorado: 1
Connecticut: 1
Delaware: 1
Florida: 0
Georgia: 0
Hawaii: 1
Idaho: 0
Illinois: 1
Indiana: 0
Iowa: 0
Kansas: 0
Kentucky: 0
Louisiana: 0
Maine: 1
Maryland: 1
Massachusetts: 1
Michigan: 0
Minnesota: 1
Mississippi: 0
Missouri: 0
Montana: 0
Nebraska: 0
Nevada: 0
New Hampshire: 1
New Jersey: 1
New Mexico: 1
New York: 1
North Carolina: 0
North Dakota: 0
Ohio: 0
Oklahoma: 0
Oregon: 1
Pennsylvania: 0
Rhode Island: 1
South Carolina: 0
South Dakota: 0
Tennessee: 0
Texas: 0
Utah: 0
Vermont: 1
Virginia: 1
Washington: 1
West Virginia: 0
Wisconsin: 0
Wyoming: 0
Calculating scores, with each set of states reduced to an array:
let sum = 0;
for (let i = 0 ; i < 50 ; i++) {
sum += (predictions[i] - results[i]) ** 2;
}
console.log(sum * 1/50);
Corrected Manifold totals:
Alabama: 0.01
Alaska: 0.08
Arizona: 0.28
Arkansas: 0.02
California: 0.99
Colorado: 0.97
Connecticut: 0.98
Delaware: 0.94
Florida: 0.07
Georgia: 0.32
Hawaii: 0.99
Idaho: 0.02
Illinois: 0.98
Indiana: 0.02
Iowa: 0.06
Kansas: 0.04
Kentucky: 0.02
Louisiana: 0.01
Maine: 0.92
Maryland: 0.97
Massachusetts: 0.9
Michigan: 0.58
Minnesota: 0.91
Mississippi: 0.02
Missouri: 0.02
Montana: 0.02
Nebraska: 0.02
Nevada: 0.38
New Hampshire: 0.85
New Jersey: 0.97
New Mexico: 0.93
New York: 0.98
North Carolina: 0.31
North Dakota: 0.01
Ohio: 0.11
Oklahoma: 0.02
Oregon: 0.98
Pennsylvania: 0.51
Rhode Island: 0.98
South Carolina: 0.03
South Dakota: 0.02
Tennessee: 0.02
Texas: 0.06
Utah: 0.02
Vermont: 0.98
Virginia: 0.88
Washington: 0.99
West Virginia: 0.02
Wisconsin: 0.54
Wyoming: 0.01
@IsaacKing It seems there is something wrong with the 538 totals too. Your record above puts 538 Arizona at 0.001 but seems to me this should be 0.31.
How in the hell did it end up with 51%... Is this the the 538 model everybody is talking about? https://projects.fivethirtyeight.com/2024-election-forecast/ Did it get un-paywalled just recently, or something?
@StepanBakhmarin No it's not paywalled, but I think it's hard at this point to see what the probabilities were 1 week before the election, since it will now be showing the frozen model from the night before.