Will an open source model beat GPT-4 in 2024?
AIOpenAIGPT-4 speculationLLMs
Basic
58
Ṁ13k
Dec 31
89%
chance

Since the release of GPT-4 by OpenAI, the AI field has witnessed rapid advancements. Open source models, leveraging collaborative and transparent development, have been a significant part of this innovation. This question seeks to forecast whether an open source model will surpass GPT-4(-1106) in terms of capabilities by the end of 2024.

Criteria for Resolution:

  1. Performance Metrics: The open source model must demonstrate superior performance compared to GPT-4(-1106) in at least X standardized major benchmarks. [Will specify later]

  2. Public Acknowledgment: The superiority of the open source model over GPT-4 must be recognized by a consensus in the AI research community, evidenced through published papers, independent benchmarking reports, or notable industry acknowledgments.

  3. Open Source Verification: The model must be verifiably open source, with its codebase freely available to the public.

Get Ṁ600 play money

Related questions

8 Comments
54 Holders
137 Trades
Sort by:

The answer here is almost certainly going to be "yes if llama counts as open source" (meta calls it open source, but eg Simon Willison would insist it's only "open weights")

Would Llama 3 be considered open source under the resolution criteria?

Looking good

predicts YES

"The model must be verifiably open source" - by "open source", do you mean just "available", or do you mean this (widely-accepted) definition: https://opensource.org/osd/

"The model must be verifiably open source, with its codebase freely available to the public." -- should also the model weights (parameters) be freely available? I recommend yes.

What does "its codebase freely available to the public" mean? Does a torrent containing just the weights (a la Mistral) count?

@dp I recommend it doesn't count, since from the weights alone, the model cannot be independently executed

Related questions