Since the release of GPT-4 by OpenAI, the AI field has witnessed rapid advancements. Open source models, leveraging collaborative and transparent development, have been a significant part of this innovation. This question seeks to forecast whether an open source model will surpass GPT-4(-1106) in terms of capabilities by the end of 2024.
Criteria for Resolution:
Performance Metrics: The open source model must demonstrate superior performance compared to GPT-4(-1106) in at least X standardized major benchmarks. [Will specify later]
Public Acknowledgment: The superiority of the open source model over GPT-4 must be recognized by a consensus in the AI research community, evidenced through published papers, independent benchmarking reports, or notable industry acknowledgments.
Open Source Verification: The model must be verifiably open source, with its codebase freely available to the public.
1,000
3.00What is even the codebase of an llm? Meta published their training code, the inference code has always been open source. If the training data is considered, this will 100% always resolve to no since no company can publish their training data for copyright reasons. Is this market just a bet on how this question will get answered?
ig you already have a large % of your nw on NO so it's reasonable not to put more. Ig this is a pretty strong case for NO since afaict Llama 3.1 405B doesn't have its codebase freely available to the public, so doesn't satisfy the resolution criteria. tbc I'm not speaking in my capacity as a mod, just confused about the market price
Looking good
"The model must be verifiably open source" - by "open source", do you mean just "available", or do you mean this (widely-accepted) definition: https://opensource.org/osd/
@dp I recommend it doesn't count, since from the weights alone, the model cannot be independently executed