I previously asked about GPT3.5: https://manifold.markets/PeterWildeford/will-i-peter-wildeford-think-that-t

Will I (Peter Wildeford) think that there is an open source LLM as good as GPT3.5 by EOY 2023?

86% chance. On Twitter, @StephenLCasper predicted: “Like what happened with DALLE2 and Stable Diffusion, I predict that within a few months, a ChatGPT copycat model will be open sourced. And then all of OpenAIs work to make their model safe will be negated by the copycats they directly enabled.” I…

In this question, I will use my subjective judgement to decide if there has been an open source LLM as good as GPT4 by the end of this year.

Grading the quality of an LLM is difficult. I'm planning to evaluate this in large part based on whether I can access this LLM and find it to be subjectively about is good, but I will also be interested in appealing to moderately standard benchmarks like MMLU.

Whether something is "open source" is defined liberally here and also will be determined by my subjective judgement, but generally I will deem something open source if (a) anyone can access it and (b) it wasn't the result of an unintentional leak/exfiltration, regardless of the precisions of the license.

I will rely on my subjective judgement to evaluate the credibility of cases. In the case this question is to resolve, I will allow 48 hours of discussion before resolving.

I will not personally be trading on this market because it relies on my subjective judgement.