. To resolve YES we need an example I can replicate in a Temporary Chat (no access to my chat history) with Thinking explicitly turned on.

I'm not trading in this market myself and will use my judgment for what counts as egregious.

PS: I extended the market close and made the FAQ explicit below instead of just referencing the previous market. The original close date of September 7th is still the cutoff. Any improvements to GPT-5 after that date don't count for a NO resolution. But we find an example of an egregious failure after that date, we'll presume GPT-5 didn't get dumber and can still resolve YES on the basis of such examples.

No, we need a prompt that can, for humans, be posed and answered out loud.

2. Does it have to commit the egregious error in response to one single prompt?

In my testing, GPT-5-Thinking gets those correct by writing and running Python code. Since it does that seamlessly and of its own accord behind the scenes, I'm counting that as the LLM answering correctly. It even evinces perfect self-awareness about its difficulty with sub-token perception and why it needs to execute code to get a definitive answer.

4. What about other questions humans would find tedious and time-consuming?

Again, it has to be posable out loud to a human. (But also GPT-5-Thinking can typically write code that solves such problems.)

5. What if GPT-5 errs but corrects itself when questioned?

That's far better than digging itself in ever deeper, but this question is about single prompts. However, if GPT-5 is just misreading the question, in a way that humans commonly do as well, and if GPT-5 understands and corrects the error when it's pointed out, I would not call that an egregious error.

6. What if the error only happens with a certain phrasing of the question?

As long as rephrasings don't count as clarifications or otherwise change the question being asked or how difficult the question is for humans to answer, then we'll consider it in the same category as 

 if the failure depends on a certain exact phrasing. So, no, it can't depend on the exact phrasing.

7. What if GPT-5 overlooks a detail in the question?

If it's a human-like error and it understands and corrects when the oversight is pointed out, that's not an egregious error.

8. What if there's high variance on how well humans perform?

Basically, if we're having to nitpick or agonize on this then it's not an egregious error. Of course, humans do sometimes make egregious errors themselves so there's some confusing circularity in the definition here. In the 

 we used "9 out of 10 people sampled literally on the street give a better answer than the AI" and I did some elaborate surveying. For this market we'll just use the median Manifold community member and I'll use my judgement. I don't want to quibble about how dumb a literal person off the street might be, like we did last time. I just need to see some example of GPT-5-Thinking being blatantly wrong in a way that none of us -- the high-quality, conscientious commenters -- would be.

In principle, yes. Namely, if we can identify a principle by which to do so.

[Ignore AI-generated clarifications below. Ask me to update the FAQ if in doubt.]