By 2028, will there be a language model of less than 10B parameters that is superior to GPT-4?
14
1kṀ553
resolved May 14
Resolved
YES
16

Pretty impressed by this:

https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/

I will not bet in this market. I will look primarily at whatever benchmarks are most prominent at the time to determine which model is better.

The model does not have to be clearly superior to gpt-4, if it's just a bit better I will resolve YES. If anyone has suggestions for less subjective resolution criteria that won't be goodharted I am open.

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ36
2Ṁ32
3Ṁ10
4Ṁ8
5Ṁ6
Sort by:

@SemioticRivalry I just noticed this question seems to have some similarity to this other one:

/_deleted_/will-a-15-billion-or-less-parameter

You seem to have a lot of knowledge about this subject and I very much do NOT. Would you be able to review my chain of comments in the other market and confirm them or refute them, and also see if they apply in any way to your own market here?

(My comments there are just completely flailing around, I have no idea what any of it means. Even though the market is resolved I am very willing to reconsider if someone knows more about the subject.)

@Eliza yeah I think this is sufficient

Gemma 2 9b-it is higher than original ChatGPT on llmsys arena. I don’t think it’s better or even on the same level but I think it’s a signal that it’s already close. 27b version I think in the same level

Related

if the model can be stored on 10b params but takes 100000x more time at inference time, because it searches through lots of possibilities or reprompts itself a bunch of times or similar tricks to improve its output, would that still count? In 2023 it would still not be practical to run, but it would still be smaller in the ‘# of parameters‘ sense

@TheBayesian I'll allow it, parameters are the only variable of interest

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules