On Twitter, @StephenLCasper predicted:
"Like what happened with DALLE2 and Stable Diffusion, I predict that within a few months, a ChatGPT copycat model will be open sourced. And then all of OpenAIs work to make their model safe will be negated by the copycats they directly enabled."
I'm personally skpetical of this prediction.
In this question, I will use my subjective judgement to decide if there has been an open source LLM as good as GPT3.5 by the end of this year.
Grading the quality of an LLM is difficult. I'm planning to evaluate this in large part based on whether I can access this LLM and find it to be subjectively about is good, but I will also be interested in appealing to moderately standard benchmarks like MMLU.
Whether something is "open source" is defined liberally here and also will be determined by my subjective judgement, but generally I will deem something open source if (a) anyone can access it and (b) it wasn't the result of an unintentional leak/exfiltration, regardless of the precisions of the license.
I will rely on my subjective judgement to evaluate the credibility of cases. In the case this question is to resolve, I will allow 48 hours of discussion before resolving.
I will not personally be trading on this market because it relies on my subjective judgement.
1,000
3.00@PeterWildeford Are you going to resolve this market? And if no, do you want to propose an alternate resolution system?
Delegate this to trustworthy users?
@PeterWildeford Have you tried any of the open-source models recently? The software LM Studio is getting pretty popular and you can try pretty much any of the open source models
Will you take LLaMA2 as open source? It's technically not, but for most people it's as if it were (unless you're a business with 700 million users as of last month)
@firstuserhere If I can personally access the weights and it's not due to some unusual characteristic of me, then yes
@PeterWildeford yes you can personally access the weights but they do ask you to fill out a form prior to that. The form asks for just the name afair. It took me 15 seconds to fill the form and i had weights within a minute after that
I made a GPT4 version: https://manifold.markets/PeterWildeford/will-i-peter-wildeford-think-that-t-c95ff3c1b385
@PeterWildeford Guanaco-65b & Guanaco-33b have already beat ChatGPT on benchmarks, you might want to try those
@PeterWildeford Huggingface: https://huggingface.co/timdettmers/guanaco-65b
https://huggingface.co/timdettmers/guanaco-33b
Demo of 33B (65B is even better supposedly)
https://huggingface.co/spaces/uwnlp/guanaco-playground-tgi
@PeterWildeford Just to be clear, if either an open source inframodel (base model) beats GPT-3.5 base or an open source tuned model beats ChatGPT, you will resolve this to YES?
@ampdot Correct
Though we should be careful to explain what "beats" means, and this will be subjective and come from a moderately skeptical approach.