Will I (Peter Wildeford) think that there is an open source LLM as good as GPT4 by EOY 2023?
resolved Jan 1

I previously asked about GPT3.5: https://manifold.markets/PeterWildeford/will-i-peter-wildeford-think-that-t

In this question, I will use my subjective judgement to decide if there has been an open source LLM as good as GPT4 by the end of this year.

Grading the quality of an LLM is difficult. I'm planning to evaluate this in large part based on whether I can access this LLM and find it to be subjectively about is good, but I will also be interested in appealing to moderately standard benchmarks like MMLU.

Whether something is "open source" is defined liberally here and also will be determined by my subjective judgement, but generally I will deem something open source if (a) anyone can access it and (b) it wasn't the result of an unintentional leak/exfiltration, regardless of the precisions of the license.

I will rely on my subjective judgement to evaluate the credibility of cases. In the case this question is to resolve, I will allow 48 hours of discussion before resolving.

I will not personally be trading on this market because it relies on my subjective judgement.

Get Ṁ1,000 play money

🏅 Top traders

#NameTotal profit
Sort by:
bought Ṁ100 of YES

Does this consider additional functionality added to GPT-4 (i.e. code interpreter or plugins) or any further updates OpenAI makes to the model during the year?

@agucova I do want to count GPT4's general ability to understand code and how to interact with APIs. So code interpreter seems fair game. But if the only relevant difference is engineering work on top of the model I won't count that. So being able to order from Instacart seems out of bounds.

Added M500 in subsidy