What is the main reason behind GPT-4o speed improvement relative to GPT-4 base model? | Manifold

What is the main reason behind GPT-4o speed improvement relative to GPT-4 base model?

AI GPT-4 speculation GPT-4

Basic

22

Ṁ1.4k

2029

1D

1W

1M

ALL

69%

Smaller model size (hence, architecture/algorithm improvements)

40%

Something related to low-level computation efficiency (for example, optimized frameworks)

27%

More/better hardware allocated

23%

Other

15%

Better coarser-grained tokenizer

Get Ṁ600 play money

Related questions

What will be true about GPT-4.5?

Will there be a GPT-4.5 model before GPT-5 is released?

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

What is true of the new Japanese GPT-4?

Is GPT-4 best? (Thru 2025)

Will the performance jump from GPT4->GPT5 be less than the one from GPT3->GPT4?

What will the aggregate improvement of GPT5 be over GPT4 in terms of metrics?

Was GPT-4 trained in 4 months or less?

Will we have an open-source model better than GPT-4-Turbo before 2025?

Will Llama 3 400B be better than GPT-4?

Sort by:

did they answer this for gpt-4-turbo?

"Main reason" implies only one of these resolves YES. If the coarse tokens help but aren't the central boost, I assume that option resolves NO?

@MaxHarms Correct. In very ambiguous situations, 2 options may resolve YES, but I expect this to happen in only extreme cases.

It's likely a combination of smaller model and low level optimisations (they are happening all the time, judging by open source solutions). However I find it unlikely that "open" AI will share exact numbers to determine what exactly played the biggest role.

bought Ṁ10 Something related to... YES

What do quantization count towards?

@Sss19971997 quantization itself would be "something related to low-level computation efficiency".

which base model?

@StephenMWalkerII The first publicly available GPT-4 model released on March 14, 2023.

Related questions

What will be true about GPT-4.5?

Will there be a GPT-4.5 model before GPT-5 is released?

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

What is true of the new Japanese GPT-4?

Is GPT-4 best? (Thru 2025)

Will the performance jump from GPT4->GPT5 be less than the one from GPT3->GPT4?

What will the aggregate improvement of GPT5 be over GPT4 in terms of metrics?

Was GPT-4 trained in 4 months or less?

Will we have an open-source model better than GPT-4-Turbo before 2025?

Will Llama 3 400B be better than GPT-4?

Related questions

What will be true about GPT-4.5?

Will the performance jump from GPT4->GPT5 be less than the one from GPT3->GPT4?

Will there be a GPT-4.5 model before GPT-5 is released?

What will the aggregate improvement of GPT5 be over GPT4 in terms of metrics?

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

Was GPT-4 trained in 4 months or less?

What is true of the new Japanese GPT-4?

Will we have an open-source model better than GPT-4-Turbo before 2025?

Is GPT-4 best? (Thru 2025)

Will Llama 3 400B be better than GPT-4?