"Large models aren’t more capable in the long run if we can iterate faster on small models" within five years
"Large models aren’t more capable in the long run if we can iterate faster on small models" within five years
4
90Ṁ229
May 4
8%
chance

LoRA updates are very cheap to produce (~$100) for the most popular model sizes. This means that almost anyone with an idea can generate one and distribute it. Training times under a day are the norm. At that pace, it doesn’t take long before the cumulative effect of all of these fine-tunings overcomes starting off at a size disadvantage. Indeed, in terms of engineer-hours, the pace of improvement from these models vastly outstrips what we can do with our largest variants, and the best are already largely indistinguishable from ChatGPT. Focusing on maintaining some of the largest models on the planet actually puts us at a disadvantage.

Giant models are slowing us down. In the long run, the best models are the ones which can be iterated upon quickly. We should make small variants more than an afterthought, now that we know what is possible in the <20B parameter regime.

Resolves to my subjective opinion of the consensus. (I don't have any better ideas)

(If you think this market is poorly written/specified/has a mistake, tell me, and I may edit it within the first couple days)

Operationalizing a claim from https://www.semianalysis.com/p/google-we-have-no-moat-and-neither (hacker news comments)

Get
Ṁ1,000
to start trading!


Sort by:
1y

sister markets - The gap will close between the quality of open source language models and Google's internal language models in two years / five years
"Large models aren’t more capable in the long run if we can iterate faster on small models" within two years / five years

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.

Related questions

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules