OpenAI has recently announced that they are training a new frontier-scale model. How much will this model cost?
By "gpt-4.5" here I mean any model that is clearly announced as a more capable newer generation of model, ideally whatever is the output of the training run they are currently performing. If they call the model "gpt-5" or "gpt-4.25" or "gpt-next" or anything like this that will count as "gpt-4.5" for the purpose of this question.
This will be an average of input and output prices per million tokens, with the formula (output price + 2x input price) / 3. I will take the pricing for the cheapest option available that is the same model (e.g. gpt-4-8k, not gpt-4-32k). On this metric claude opus is $35/mtok, gpt-4 is $40/mtok, gpt-4-turbo is $16/mtok, and gpt-4o is $8/mtok. I will go by the first publicly announced prices I see.
Possible clarification from creator (AI generated):
The O1 line of models will not count as they are not considered a new base model
This market is intended to capture the price of the "Orion" model or equivalent new base model that represents the output of OpenAI's current training run
Update 2024-16-12 (PST) (AI summary of creator comment): - The market will only resolve based on a new base model/pretrain, not RL tuned versions
O1 models are considered RL tunes of GPT-4 and will not count for resolution
1,000
3.00@sophiawisdom does this resolve with the O1 line of models?
O1-mini was announced at the same time as o1-preview, and o1-preview has a pricing of $15/mtok input and $60/mtok output, so (60 + 15*2)/3 == $30 on the dot, which I presume means this market resolves to the 30-60 bucket (since the first bucket is <30?).
I'm a little surprised I'm the first to find this.
edit: (no longer relevant)
@Sketchy Interesting point. I think not, because in my understanding o1 is not a new base model. It's something I should have accounted for when I asked this question. As I wrote, I was trying to capture "ideally whatever is the output of the training run they are currently performing" -- I believe this to be the "Orion" model they are likely planning to announce in the next 9 days.
@sophiawisdom Gotcha. Do you think they were training Orion back in early June? That quote about "whatever the output of the current training run" was honestly more evidence to me that you meant o1, since it was released in September, a couple months after this market was created on June 3.
How would you have accounted for it when you wrote the market, since it wasn't released yet?
@Sketchy I was aware that openAI was in the middle of training a new large model. I think there were news articles about this at the time. I was referring to a new base model / pretrain. O1 is, afaict, a RL tune of gpt-4o
@wrhall I hadn't considered this question. I suppose I mean the most expensive (presumably highest quality) model that they release. Thank you for the clarification.
I would note that as far as I can tell there haven't been two SOTA models released at the same time before.
@sophiawisdom this is $5/million input, $15/million output for gpt4o
https://openai.com/api/pricing/
I think this can resolve?
@chris I don't count gpt-4o as gpt-4.5, given that it is inferior to gpt-4-turbo on a number of benchmarks (or at least not clearly superior). What I am primarily intending to capture in this market is the new frontier-scale model that OpenAI announced they were training recently. I listed gpt-4o in the description, which I think makes it clear I didn't count gpt-4o.