Similar to /LoganZoellner/will-we-have-a-foss-equivalent-of-g
With this much excitement and energy in the LLM space, I am fairly confident in this prediction: Before the end of this year, there will be an open source model that is equivalent in quality to #GPT4
Resolves positive if a FOSS model (no signup required to download) is pubically available by 12/31/25 that outperforms GPT-4 on at least 50% of the tests listed in this article (https://www.semafor.com/article/03/15/2023/how-gpt-4-performed-in-academic-exams)
Could make sense to base the resolution criteria off of ML benchmarks instead of academic/professional exams, since most open source models have not been evaluating on these exams, and there are not very many details on the specific exams/questions used. Some common benchmarks are HellaSwag, MMLU, and the AI2 Reasoning Challenge, which are listed in the GPT4 technical report, and I am using in my market: https://manifold.markets/tk/will-meta-release-an-open-source-la
