Will an EU-based AI lab announce a chatbot of GPT-3.5's capabilities in 2023?
closes Dec 31

Will resolve positively if a company registered in an EU country announces a self-developed LLM-based chatbot of roughly the capabilities of ChatGPT, 2022 version (OpenAI), Bard (Google), or Claude (Anthropic) by the end of 2023.

Get Ṁ500 play money

Related questions

Will OpenAI hint at or claim to have AGI by 2025 end? (8000M subsidy)
firstuserhere avatarfirstuserhere
23% chance
Will Anthropic automate AI research in 2024?
SneakySly avatarSneakySly
9% chance
Will a major AI lab announce that the weights of one of its models were compromised in a cyberattack before 2027?
AdamK avatarAdamK
50% chance
Will AI agents be used to develop software commercially by the end of 2023?
AlexMizrahi avatarAlex Mizrahi
69% chance
By the end of 2026, will we have transparency into any useful internal pattern within a Large Language Model whose semantics would have been unfamiliar to AI and cognitive science in 2006?
Will OpenAI release a search engine before 2024? [Read description]
FranklinBaldo avatarFranklin Baldo
61% chance
Will there be an AI language model that surpasses ChatGPT and other OpenAI models before the end of 2024?
Gen avatarGenzy
57% chance
Will Tyler Cowen agree that an 'actual mathematical model' for AI X-Risk has been developed by October 15, 2023?
JoeBrenton avatarJoe Brenton
8% chance
Will it be public knowledge by EOY 2025 that a major AI lab believed to have created AGI internally before October 2023?
dmayhem93 avatardmayhem93
15% chance
Will there have been a noticeable sector-wide economic effect from a new AI technology by the end of 2023?
Nostradamnedus avatarNostradamnedus
13% chance
Will Science's Top Breakthrough of the Year in 2023 be AI-related?
dp avatardp
34% chance
Will a 10B parameter multimodal RL model be trained by Deepmind in the next 12 months?
BionicD0LPH1N avatarBionic
66% chance
Will A.I. Become Significantly Better at Drug Discovery in 2023?
PatrickDelaney avatarPatrick Delaney
34% chance
Will there be another major public-facing breakthrough in AI before March 31, 2024 [subjective - 1000M subsidy added]
DylanSlagh avatarDylan Slagh
52% chance
Will commercial AI tutoring services become prevalent in the US by 2025?
AdamK avatarAdamK
66% chance
Will more than 20 organizations publicly train large language models by 2024?
Will AI be able to accurately do my taxes by EOY 2026?
PeterWildeford avatarPeter Wildeford
59% chance
Short Term AI 2.5: By January 2024, will there be a usable, general AI assistant?
vluzko avatarVincent Luczkow
54% chance
Will OpenAI + an AI alignment organization announce a major breakthrough in AI alignment? (2024)
Mira avatarMira 🍎
78% chance
Will artificial superintelligence exist by 2030? [resolves N/A in 2027]
Sort by:
ShadowyZephyr avatar

“roughly the capabilities of ChatGPT, 2022 version (OpenAI), Bard (Google), or Claude (Anthropic) by the end of 2023.”

So which is it? These bots all have different capabilities. GPT has the best logic/reasoning, Claude is best at creative tasks, and Bard is significantly worse than the other two.

And if the model is private, made before 2023, but gets released after, how does it resolve? Luminous-world is a potential candidate for this

5 replies
konstan avatar
Konstantinpredicts NO

@ShadowyZephyr Either counts

konstan avatar
Konstantinpredicts NO

@ShadowyZephyr Known as of dec 31

ShadowyZephyr avatar

@konstan And what about the first question? WHAT capabilities, exactly? If you're talking about any benchmark, luminous-supreme should be sufficient to resolve this to YES, because the company behind it published benchmarks showing it matching text-davinci-003 (similar performance to ChatGPT) although I personally find it way less competent than these.

konstan avatar
Konstantinpredicts NO

@ShadowyZephyr I think it's going to be a mix of benchmarks and me just reading what others think about the models

konstan avatar
Konstantinpredicts NO

@ShadowyZephyr If you're not convinced by luminous-supreme, I likely won't be either.

konstan avatar
Konstantinpredicts NO

@Tegwick I mean developing the base model from scratch, just like Google's Bard or OpenAI's GPT-3. Finetuning does not count.

What do you mean by "modified engines with training"?

6 replies
Tegwick avatar

@KonstantinPilz by the engine I mean the code for algorithms of training the model and of retrieving and tuning requests, as well as context setting, API design etc without actual weights. And I would find building from scratch hard to define still. My best effort to make building from scratch a decidable criterion is having a code base which does not reuse stuff from another model but even then you may or may not allow use of common libraries. And if it is fully or in part the same algorithms would one call it "developed from scratch" because the code istl different?

konstan avatar
Konstantinpredicts NO

@Tegwick Fair!
Let's say developed from scratch means they trained it. Fair to use already-existing code (though, afaik, neither Google nor OpenAI nor Anthropic has published it). It's about whether or not they are capable of a large-scale, long training run required to obtain one of those models.

JustNo avatar

@Tegwick if this includes retraining modified open source (right now that mostly means LLaMa derived) models, I think it's substantially undervalued.

Can you confirm that a research institution doing that would qualify?

Relevant google leak: https://www.semianalysis.com/p/google-we-have-no-moat-and-neither

JustNo avatar

@JustNo above comment/question intended for @Konstantin

konstan avatar
Konstantinpredicts NO

@JustNo I'm interested in whether an EU-based lab has the ability to train something like GPT-3.5 from scratch. So no, this doesn't count (and afaik LLaMA-based models aren't at 3.5's capabilities yet)

ShadowyZephyr avatar

@konstan Vicuna is quite close. I personally would put it under gpt-3.5-turbo, but there are some questions that it's better at.

Tegwick avatar

What are companies or research facilities that might do that? What candidates are there? What does self developed mean? I guess using an existing model/engine and training it does not count. But what about modified engines with training. There is no specific criterion that I can easily use to decide how self-developed should be defined.

MarkIngraham avatar
Mark Ingrahambought Ṁ5 of NO

Britain is not eu