I don't know what people's expectations were, exactly, but this is my take:

It seems pretty clear that GPT-5 was not a GPT-3.5 -> GPT-4 moment, but it's good enough at pair programming that it has stolen me back from Gemini 2.5 Pro.

All that for +21 Elo score?

@TheAllMemeingEye bruh it's only 257 above GPT-3.5 from way back in 2022-23, so almost a fifth of the time the model from years ago will beat it XD

I have a take:

It's just o3 RL'ed to have variable thinking. It doesn't feel like a new model at all

@FergusArgyll what does variable thinking mean in this context?

@TheAllMemeingEye It (or a classifier) chooses how much reasoning to devote to a query.

Not an actual technical term as far as I know

If anyone wants to keep trading https://manifold.markets/Thomas42/how-will-gpt-5-exceeds-expectations

EDIT: Poll will be posted in one month (per description)

lmao those are some interesting numbers with that chart

@Balasar chart made by GPT-5

@Balasar lol

@Balasar Holy shit that's misleading lol

Closed so no arbitrage possible

https://manifold.markets/PhilosophyBear/when-gpt5-comes-out-will-more-manif

Is there a similar market, but for GPT 5 being underwhelming?

https://manifold.markets/strutheo/will-nvidia-stock-nvda-be-worth-188
This market should be correlated with this one, no?

@jessald Very loosely at best, I hardly expect that the performance of one model is going to have a big effect on Nvidia's stock

Another factor to consider is the demographics of the voters in the future survey.
I wonder if they might be biased?

let’s roll

In an interview Dario said that the average lay person wouldn't be able to appreciate an AI with for example PhD level chemistry knowledge because they simply wouldn't understand or even ask for information that would make use of or reveal this knowledge I feel like gpt5 could be an iceberg and most of the respondents in the future poll will only see the surface.

@jessald Everyone knows something & they could test it on that. That works if you're a car mechanic too. Besides, you'd hope to see better instruction following

@jessald Dario's point is 100% true in theory but in practice I feel the opposite has happened, at least in some forums: people unable to actually judge models' performance in, say, advanced mathematics, go around claiming models are way better than they actually are.

My read is that people are generally expecting a qualitative jump similar to the jump from GPT 3.5 to 4, but that they will use 4o or o3 as the comparison, rather than the original 4.

@NBAP They should use 4.5, no?

Everyone just memory-holed 4.5! It's such an awful model, I keep trying to use it and it really can't hold a candle to o3 and it's just as slow.

sometimes i hate taking EV bets like this but w/e. this should obviously be about near 50%.

My general thought process is that anyone who reads METR works should say no unless we have reason to believe capabilities have escalated beyond the METR scaling we previously thought possible.

@No_uh In principle yes ... but expectations are pretty vibe based, we are not living in a nation composed of Bayesian rationalists here

@BorisBartlog ya but this will be resolved according to manifold poll, and also gpt-5 reactions will be a multi-national thing. anyway i consider the likelihood people vote according to their profit opportunity here a much larger complication than the unawareness or awareness of how closely it aligns with METR scaling projections

@No_uh Yeah I can see that being a problem, the YES:NO ratio is far from even