@traders I resolved the market. Please find a summary of my thinking below.

Can gpt-4o be considered openai's next-generation model (e.g. 4.5 or 5)?

No, because

1. intelligence leap not comparable to gpt-3 to gpt-4 jump

   • various metrics show smaller improvement

   • incremental rather than revolutionary advancement

2. lack of official next-gen designation from openai

   • announced "frontier models" coming soon at end of presentation

   • model's efficiency and speed suggest it's not dramatically larger

   • 4 in name implies iteration on gpt-4, not new generation

Yes, because

1. next-gen status is not solely determined by model size:

   • llama 3 8b is considered next-gen despite smaller size

   • anthropic's claude 3.5 sonnet is next-gen and most intelligent, but smaller than opus 3

   • industry trends shifting towards viewing model families rather than strict generational leaps

2. significant capabilities and advancements:

   • multimodal integration: text, audio, and vision capabilities

   • improved efficiency and lower operational costs

   • not merely a fine-tuned version of gpt-4, but architecturally distinct

3. released through a major event:

   • launch signifies importance and marks a significant milestone

   • demonstrates openai's commitment to positioning it as a major advancement

Ultimately, there's no right or wrong way to answer this question. At the end of the day, these are all marketing terms. However, based on everything I listed above, I personally think of GPT-4O as next-generation. While it doesn't represent as dramatic a leap as GPT-3 to GPT-4, it introduces significant new capabilities and efficiencies. It's likely more of a 0.5 jump than a full generational leap, aligning with the recent release of Claude 3.5 Sonnet. It seems that moving forward a nuanced view of generations is needed. You can get a new generation of a small model. "Next-generation" might not always mean vastly larger or more intelligent, but rather more capable, efficient, or versatile. GPT-4O embodies this trend, making a strong case for its consideration as a next-gen model despite counterarguments.

Is it time

@ismellpillows i agree it’s time to resolve this. right now, i lean towards gpt-4o is openai’s next-gen model. it’s not the largest, and openai will probably release a bigger model this year. but i think 4o and any new model would be part of the same family, like claude 3 with opus and sonnet, or llama 3 with the 70B and 400B models. gpt-4o is more efficient, cheaper, supports new modalities, isn’t just a fine-tuned version of gpt-4, and was announced through a major event.

I think your 50/50 proposal is more fair. GPT4o has that snazzy "4" in its name - it's obviously not the "next generation" after GPT4.

@traders i will resolve to gpt-4o is openai’s next-gen model in 24 hours if nothing changes

@traders I did not expect this to happen, but what do you all think about potentially resolving this market as 50% "OpenAI's next generation model" and 50% "other (e.g. just an updated version of gpt-4)"

@chrisjbillington FYI

@Soli or resolve 2 variants to 0, and the other to to 23 and 77 (proportionally to the market value)

@KongoLandwalker this would be an option yes. What would be the reasoning behind doing this?

@Soli No. The description distinguishes "next generation model" from "finetuned version of old model".

gpt-4o is stated to be a new model. So it's not properly an updated version of GPT-4, and is most closely a "next generation model".

@Mira the description was specifically referring to gpt-2 in that case but I definitely see where you are coming from. If I have to choose only one option, then "next generation model" would be more accurate but this doesn't mean that gpt-4o can't also be an "updated version of gpt-4"

I think the problem stems from the fact that most of the new stuff in gpt-4o is audio/vision capabilities and usability. If you only judge the model based on text output, then it is very close to gpt-4.

I didn’t consider this scenario when I created the question, which is why we ended up with two options that can both be true. Does this make sense to you?

@Soli the description has this part though which would be problematic for a 50/50 resolution

I will resolve the market to the option most closely resembling the truth.

@Soli I am biased, but there probably won’t be a GPT-4.5 (see related markets), so this is what the next generation model is called (by OpenAI branding).

But I’m also ok with 50/50 given ambiguity

A good counterpoint is this market refers to gpt2-chatbot which, depending on definition, is functionally GPT-4

interesting slide from OpenAI

https://twitter.com/rowancheung/status/1790066757115302381?s=46

@WalterJr https://twitter.com/sama/status/1790066003113607626

@traders i will close this question till i have time to clarify some stuff

This can resolve!

https://x.com/LiamFedus/status/1790064963966370209

@jackgwhit strictly speaking that tweet is about "im-also-a-good-gpt2-chatbot" and this market is about "gpt2-chatbot", and although the models are presumably related it's not clear in what way.

@chrisjbillington yeah perhaps, but i view the evidence as incredibly strong. curious what else will come out!

@traders there is a strong discrepancy between this market and /Soli/is-this-real-gpt45 20% vs 6%

Interesting that karpathy has been doing so much open source work on gpt2 since leaving openai

@RemNi he is :)

I'd bet GPT-2+Q* if the resolution was more objective and not in a month.

@Bair I made one that goes until end of year. Feel free to suggest improvements to the resolution criteria. https://manifold.markets/jim/is-gpt2chatbot-gpt2

Where do I answer "A Small Language Model"? I think it's just OAI flexing how well they went under gpt2 parameter architecture.

@MP If that's the case it's a breakthrough of two or more orders of magnitude. gpt2-chatbot It performs similarly to models in excess of a hundred billion parameters, while GPT-2 had 1.5 billion