Will Nate Silver's 2024 election model be more accurate than 538's model?
Will Nate Silver's 2024 election model be more accurate than 538's model?
Basic
31
Ṁ1800
resolved Apr 22
Resolved
N/A

Nate Silver, the founder of FiveThirtyEight, recently was laid off (it has been a part of ABC for the last decade). According to him, "Much of FiveThirtyEight’s vital intellectual property — such as the election forecast models — is merely licensed to Disney. The license term for these models expires with [Nate Silver's] contract this summer. [He] still own these models, and can license or sell them elsewhere" (https://natesilver.substack.com/p/some-personal-news). If he releases predictions for the 2024 US elections, will they be more accurate than the model put out by FiveThirtyEight?

If either Nate Silver or FiveThirtyEight do not release election predictions for 2024, resolves N/A. If Nate Silver continues to license his model to Disney/ABC/FiveThirtyEight, resolves N/A. If Nate Silver licenses his model to some group other than Disney/ABC/FiveThirtyEight, that will count as "his model".

The predictions used will be the final predictions released before the elections. Any predictions given earlier than that will be ignored, even if they are more accurate.

If multiple models are used (such as a "Deluxe Model" and a "Classic Model"), it will be whichever model is more prominent in publications. If multiple models seem to be equally prominent, it will be the model that includes more information.

I'll define "more accurate" by scoring both predictions with a "log score" method, as detailed in this Substack post (by @jack), weighted so that a Senate seat is worth 12.8 House seats and the Presidency is worth 435 House seats. This allocation is so that the House, Senate, and Presidency are all weighted roughly equally. If a model does not call a specific seat, that will count as a 50/50 prediction. A prediction phrased ">99%" or "<1%" will be treated as 99.5% and 0.5% respectively.

Get
Ṁ1,000
and
S3.00


Sort by:

Nate Silver just announced that he will be doing a 2024 Presidential model, but not Congressional. Do you think this should:

  • Resolve N/A?

  • Only count the Presidency predictions?

  • Count all House and Senate seats as 50/50 predictions for Silver (this market will almost definitely resolve NO)?

  • Leave it as-is and re-ask, hoping that he changes his mind?

I'm going to put comments for each option, and I'll go with whichever option gets the most likes.

8mo

Resolve N/A?

8mo

Only count the Presidency predictions?

8mo

Count all House and Senate seats as 50/50 predictions for Silver (this market will almost definitely resolve NO)?

8mo

Leave it as-is and re-ask, hoping that he changes his mind?

8mo

As much as I think only counting the presidency would advantage me, I think the fair thing is to N/A if he doesn't change his mind and release all three.

1y

https://www.natesilver.net/p/the-election-model-is-a-little-too/

Nate Silver is saying that

  • He might not release an election model for 2024

  • He might do a lo-fi model, with less frequent predictions or rounded predictions

  • He might sell or license it for private usage (eg. by financial firms looking to hedge risk)

In the first and third case, this would resolve N/A (since there’s no model that I would have access to judge). In the second case, I would try to compare them as similarly as possible. However, if they’re too dissimilar and meaningful comparison isn’t possible, I would resolve N/A.

To be clear, I support Silver in whatever choice he makes. It’s ultimately his personal decision and he’s completely correct about a lot of the negative effects of his model (both on himself and a lot of the public).

1y

Also, if he puts it behind a paywall, and the paywall is semi-reasonable (ie. on the scope of tens-hundreds of dollars for the election season), then I’ll pay for access. If it’s much more than that, I’m not going to pay.

predictedNO 1y

@Gabrielle In the past, 538 has described races as having a "<1%" chance of having a certain outcome. How will you treat predictions of this form?

1y

@BoltonBailey What would you suggest? I’m inclined to just treat it as 0.01 or 0.99.

predictedNO 1y

@Gabrielle My inclination would be 0.5%, but your way makes sense too. If I make any markets to arb against this I'll lean towards following whatever you do.

1y

@BoltonBailey I’ve gone with 0.5%/99.5%.

predictedNO 1y

This is one of those weird meta-forecasting questions where it's less about "how good are the respective forecasters" and more about "what is the chance that Silver will be more uncertain about the outcome XOR what is the chance that the favored party the day before the election will outperform expectations."

predictedNO

I think it's maybe likelier that Silver will be more uncertain, but the other question is 50/50 almost by definition and unlikely to be correlated, which I think makes this market pretty close to 50/50. Maybe there are some effects around the presidency being an all-or-nothing thing.

1y

@BoltonBailey Relevant question: If we backtested this on the 2020 election, where democrats underperformed polls but Biden was still elected, would a predictor who was slightly more bullish on democrats have been more accurate according to this metric?

predictedNO 1y

@BoltonBailey If anything, maybe this market should be at less than 50% on the grounds that Silver is likelier to be more uncertain, but that whatever 538 and he both predict is likelier will come true.

1y

You should judge the models by how well calibrated they are, not whether they "correctly" predict binary outcomes. For example: https://firstsigma.substack.com/i/85825582/scoring

predictedYES 1y
1y

@cos Good suggestion, I’ll try to use the same log score as that Substack post. Thanks!

1y

(For future context, the scoring model was originally binary for each race; see the edit history for more information)

1y

@Gabrielle Yeah, I think Nate’s will probably be “better” (85%), but in any given election, there’s correlated variance, so that’s why I bet NO.

1y

@cos I don't think calibration is the only factor. If I predict every seat at 50% my calibration will be great, but I won't be a good predictor.

1y

@cos Ironic that the real money ones are less accurate.

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Win cash prizes for your predictions on our sweepstakes markets! Always free to play. No purchase necessary.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like trading still use Manifold to get reliable news.
How do I win cash prizes?
Manifold offers two market types: play money and sweepstakes.
All questions include a play money market which uses mana Ṁ and can't be cashed out.
Selected markets will have a sweepstakes toggle. These require sweepcash S to participate and winners can withdraw sweepcash as a cash prize. You can filter for sweepstakes markets on the browse page.
Redeem your sweepcash won from markets at
S1.00
→ $1.00
, minus a 5% fee.
Learn more.
© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules