This market resolves based on my subjective judgement. I won't bet. Its close date is either immediately after an explanation is given, or a week after a model is released (explanation or not) or 538 announces they'll never release a model (explanation or not). If they literally never say anything at all, resolves to "no explanation given" on Nov 5.
This market will resolve 100% to all excuse(s) given by 538 or their direct representatives regarding why the model is so delayed. G Elliot Morris on Twitter counts, as would a statement by ABC/538, but speculation by unaffiliated people doesn't. Feel free to suggest options I haven't added and I'll add them.
I will not resolve to "no explanation given" unless all other options resolve NO.
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ724 | |
| 2 | Ṁ316 | |
| 3 | Ṁ303 | |
| 4 | Ṁ158 | |
| 5 | Ṁ119 |
Just had time to read the article. The article itself means "No explanation given" resolves to NO, as this is clearly a discussion of why the model was not released until now, even if they don't explicitly say "we delayed it because of these things".
The other options, based on the article:
Resolves to YES:
Model revamp to account for unique circumstances: Most of the article is a discussion of the revamp they did because of the polls/fundamentals divergence. This is a clear YES.
Waiting for "official" nominee announcement at DNC: The article starts by saying they released it the day after the convention. The sentence is not explicitly causal, but neither is anything else in the article with respect to the delay, so I believe it's sufficient to resolve YES.
Technical issues / bugs: Much of the article is a discussion of issues with the original model that meant it warranted a redesign, not just because of unique circumstance but because of core flaws in design. I was initially unconvinced, but have come round to treating this as a technical issue.
Resolves to NO:
Waiting for more polling data: No mention of this.
Internal drama: No mention of this.
Well, this is a very different interpretation than mine.
technical issues / bugs: I agree with your argument that bad design is not a bug, but the option seemed to imply that either technical issues or bugs would resolve YES, and bad design does for sure sound like a technical issue to me.
waiting for official nominee announcement at DNC: The preamble mentioning the relative dates read like pure unsubstantial flavor text to me. Besides, they've previously communicated on the suspended model page that this is a not a reason they'd delay: "We will publish an election forecast including the new presumptive Democratic nominee."
To further support the first point, here are three recent news articles that use the phrase "technical issues" to mean problems with design or architecture:
@bence The more I think about it the more I agree with you on technical issues. It's not just a redesign of the model, there were issues with the original and I think that does count albeit not exactly in the way I initially envisioned with that answer.
The nominee one I still feel firmly is a YES. None of the article explicitly calls out the delay, it's right at the top and while the language isn't causal none of the article is, so I feel it's better to be consistent - unless I'm resolving to no explanation given for everything, which definitely doesn't feel right.
I've asked @mods in discord (and now here, please!) to reresolve the "technical issues/bugs" answer to YES.
I really disagree with the “technical issues” interpretation.
I’ve elaborated my reasoning in great detail on another market, but there’s no evidence at all of any technical issues.
Just because a model is BAD or yields unexpected results, doesn’t mean there’s explicitly an error or technical issue. Nowhere in their explanation is there any indication that their model was technically flawed (in fact, they even explicitly say at one point that it ran exactly as it was supposed to “in theory”).
Having flawed assumptions built into their model is NOT a technical error. @Sketchy
@benshindel Let's try another framing. The popular sentiment was 'the new model is delayed because they think the old one's results were wrong'. I personally also thought this would be the most likely answer (although I predicted no explanation given when the market opened, assuming they just wouldn't talk about it). If you thought they didn't trust their own results and wanted to make technical changes to improve the results, which option would you have picked? I think the only reasonable one is the technical issues one.
@benshindel yeah, I mean I obviously sympathize with you here, but ultimately, I’m convinced that if something doesn’t work in practice and requires significant technical explanation in order to convey, and they wanted to change it, it is a technical issue. There’s clearly subjectivity here, and I initially shared your opinion, but I’m comfortable with YES as the resolution. Apologies the subjectivity didn’t go your way this time.
From reading the article, it also seems like the implementation of their model didn't correspond to the intent behind their model.
How exactly you estimate that movement in every state, though, turns out to be very complicated, and somewhat more of a decision about programming rather than modeling
I think you could interpret this in 2 different ways:
1. "We couldn't translate an arbitrary mathematical model into code (maybe for tractability issues?), so we made compromises designing the model and those compromises resulted in results that don't match our intuitions"
2. "Our model is fine, but the way we implemented it didn't match our intentions because we made compromises (computational tractability? Code simplicity? Straight up bugs?) that resulted in a mismatch"
I think a reasonable definition of "technical issues/bugs" would probably be "Our code didn't match our intentions". I think that's pretty clearly true if (2) is correct, but if (1) is correct I think it's iffier? In the (1) case it's hard to draw the line from where intention ends and implementation begins.
I honestly think that (1) is what they're trying to convey here, and it's hard for me to have a firm answer on whether that's technical issues or not. It's weird, but I would probably lean towards this is a "technical issue" (because their weight on fundamentals was higher than intended due to how they decided to implement covariance in practice), but not a "bug" (because the code works as intended, just the design decisions that went into that code don't reflect their intentions)
But this is very dicey to me all around. But the sentence above chooses to frame the issue as a "programming" one (which to me belies... I dunno, something not good about their overall approach here), rather than a methodology one, which to me supports the "technical issues" interpretation.
I just think it’s not really subjective. The term “technical issues / bugs” connotes something completely different than what happened!
This is an unlinked binary, therefore no option “had to be picked”.
An option like “flaws in the model” or “methodological issues” or something should clearly resolve YES. But “TECHNICAL issues” were clearly absent. If there was, say, a bug or technical issue they would have stated that in the article. They did not do so! It should resolve NO pretty unambiguously and I don’t understand the lobbying effort to resolve that YES. To me, that just implies a lack of understanding of how these sorts of models work by the market creators, and they should defer resolution to people who have been following this for years!!
When you say "people who have been following this for years", I'm genuinely trying to understand what your claim is. I've followed 538 since the 2016 election and my professional work is loosely adjacent to different applications of similar models to the ones 538 uses (though I'd never claim to be an expert, my interaction is basically just understanding the surface level enough to sanity check outputs from the data scientists who actually understand it). Am I in the subset of people that one should defer resolution to?
Separately, I posted on my other market that I kinda envision 538's process to be something like:
1. We have a sense for what a good model would look like
2. We plan out a model in the abstract with high level ideas
3. We concretely plan out how that model will work mathematically
4. We implement that plan in code
I would say it sounds like the disconnect is between 2 and 3 here. I wouldn't call that a bug, but I think it's fair to call it technical issues if you count the statistics as a technical thing one could have issues with. They're basically saying "We had a sense for how we wanted intra-state correlation to work, but the way we modeled it mathematically in practice was more aggressive than we set out to do". I think it's fair to call that a technical issue. But I agree this is feels dicey and really hinges on how you define "technical issue".
https://abcnews.go.com/538/538-adjusting-election-model-harris-versus-trump/story?id=112563822
I think this should resolve as technical issues, and unique circumstances, per these two sentences.
These changes address issues that became apparent while running the model for President Joe Biden and Trump in June and July. We have also adjusted our model to make sure it could properly handle cases when major third-party candidates, such as Robert F. Kennedy Jr., drop out of the race.
https://www.semafor.com/article/08/25/2024/538s-model-mystery
The plot thickens! Just to be clear, this is not resolvable because it's not from 538. If they don't say anything by the end of the month, this market resolves to "No explanation given".
Forecast is back up: https://projects.fivethirtyeight.com/2024-election-forecast/
Unless I missed it, I don't see any explanation for the delay, so the clock is on for a week for 538 to offer an explanation, either on the website or via representatives on Twitter.
It seems like they panicked and changed the model assumptions because they didn’t like the outputs. I have no idea how they will spin this (Nate Cohn does not work for 538)
We will publish an election forecast including the new presumptive Democratic nominee, when such nominee is announced.
...
If Harris does become the clear presumptive nominee for the Democrats, 538 will launch a new election forecasting and polling-average model for her campaign against Donald Trump
From the wording on their website they've made it pretty clear that they're fine launching a model for a presumptive nominee, so it'd be weird to back out of that now.