Will any LLM have roughly GPT-3-level losses with a context window of at least 50,000 tokens before April of 2024?
21
139
410
Apr 1
80%
chance
Get Ṁ1,000 play money
Sort by:
bought Ṁ30 YES

@IsaacKing Doesn't Claude 2 fit this description? Performs better than GPT-3 on pretty much every task, and has a long enough context length?

I think this question is ill-posed and you shouldn't compare losses between models. It is technically well-defined(i.e. somebody with access to OpenAI's dataset could score the prediction accuracy of any model), but in practice people use standardized benchmarks like the recent GPT-4 release. And the actual loss numbers you see reported in charts won't be comparable.

I think you should N/A this market as not meaningful, or specify some benchmarks instead.

@Mira Any suggestions?

I don't actually know what "losses" means in the context of LLMs. I've seen the term before, but never looked into it. I'll learn before resolving the market of course. If anyone wants to link me to an overview, I'd appreciate it.

bought Ṁ5 of NO

Seems technically feasible, but you'd only do it if you were already training an ungodly huge next-gen model. And those are usually spaced out a bit.