Which prediction platform will be most accurate in forecasting the 2022 midterms?
19
86
resolved Jan 1
12%
Manifold
17%
538
18%
Polymarket
17%
PredictIt
19%
Election Betting Odds
17%
Metaculus

I will score several prediction platforms on a set of 10 questions on the outcome of the 2022 US midterm elections. For each prediction platform, I will take the predicted probabilities on Monday evening, and compute the average log score (a measure of the prediction's accuracy) on these questions.

This question will resolve to the platform with the best (highest) average log score.

The questions and scoring methodology are described here:

https://manifold.markets/post/comparing-election-forecast-accurac

See related questions here: https://manifold.markets/group/election-forecast-comparison

Note that the election results are highly correlated, so the platform that turns out to be most accurate may not have actually been the best set of predictions. The forecast that scores best is probably going to be the forecast that happened to best predict the broader question of how left or right skewed the entire election was, but some of that might be "luck" which it might not be able to repeat across many different election years. To truly measure accuracy well, we'd need to run this experiment several times over different election cycles.

I've selected this set of 10 questions:

  • Senate control

  • House control

  • Senate races

    • Pennsylvania - Mehmet Oz R vs John Fetterman D

    • Nevada - Adam Laxalt R vs Catherine Cortez Masto D

    • Georgia - Herschel Walker R vs Raphael Warnock D

    • Wisconsin - Ron Johnson R vs Mandela Barnes D

    • Ohio - J. D. Vance R vs Tim Ryan D

    • Arizona - Blake Masters R vs Mark Kelly D

  • Governor races

    • Texas - Greg Abbott R vs Beto O'Rourke D

    • Pennsylvania - Doug Mastriano R vs Josh Shapiro D

I will also open a series of related questions on various head-to-head comparisons and the accuracy scores of each platform.

Please let me know if you have any feedback on the design of this prediction comparison! I may make adjustments for the other questions.

Close date updated to 2022-11-08 4:00 pm

Get Ṁ1,000 play money

🏅 Top traders

#NameTotal profit
1Ṁ151
2Ṁ49
3Ṁ30
4Ṁ3
Sort by:

@jack Are you planning to do another such analysis for the 2024 US election? And if so, are you planning to do another version of this market and the other markets you created for the midterms?

bought Ṁ5 of Polymarket

I'm curious about Manifold's Salem tournament as well.

bought Ṁ5 of PredictIt

@MichaelWheatley Great idea! I'll compare them as well.

Which prediction platform will be most accurate in forecasting the 2022 midterms?, 8k, beautiful, illustration, trending on art station, picture of the day, epic composition

bought Ṁ5 of Polymarket

Tomorrow I will open a series of related questions on various head-to-head comparisons and the accuracy scores of each platform.

Please let me know if you have any feedback on the design of this prediction comparison! I may make adjustments for the other questions.

bought Ṁ15 of 538

Blake Masters vs Mark Kelly is a senate election

@Conflux Thanks, fixed, copy-pasted in wrong place