On Oct 14 2023, Yann LeCun (Chief AI Scientist at Meta) stated: "Open source AI models will soon become unbeatable. Period."
Resolves YES if, at the end of 2025, it's decisively clear (in the judgment of Eliezer Yudkowsky) that open-source LLMs (or their successors in the role of widely used AGI tech) are more powerful or more cost-efficient than their closed-source alternatives. That is, if either all the leaderboards are full of open-source LLMs with successors to GPT-4 or Claude being far behind, or if most of the business spending for inference seems to be on running AI models built on open-source foundation models, this resolves YES.
If it's hard to tell or if that seems wrong, resolves NO. "Unbeatable" seems like it shouldn't be subtle.
Related questions
@osmarks it's obviously not, but Gemini Ultra being barely better than GPT-4 arguably shows that having enormous resources and data doesn't really give you that much of an edge.
@BairAiushin My reading of that is just that Google is bizarrely incompetent some of the time.
@osmarks Google had an extra 9 months over GPT-4 + some open source knowledge, and still failed to make the gains we expected, even with delaying Ultra. This definitely is good news for YES betters here. Plus, open source is getting closer to gpt-4. I'm updating a bit towards open source seeing that it is progressing faster.
@ShadowyZephyr I haven't seen any open source models get remotely close to GPT-4, the best ones are just barely starting to get close to 3.5 Turbo.
@SemioticRivalry this is true but if the other models are going to slow down they have 2 years to catch up. In the scenario where everyone is blasting full speed ahead them catching up would not be nearly enough.
@VAPOR Not really correct, no. Threshold is based on compute, so in theory a hypothetical Llama-3 would have the reporting requirements even if then released as open source.
(I don't think those reporting requirements are a threat.)
@mariopasquato US regulators tend not to look kindly on, “One Weird Trick to avoid US regulation” gambits. Most entities training such models want to do business in America
@PatrickDelaney I am very curious as to the kind of life you have lived such that "Drinking one's own piss to stay alive" was the first analogy that came to mind here.
@MartinRandall Even in this non-conditional (non policy based) market, one should also signal their true beliefs to politicians! You're making them less likely to act based on not even profitable, but wishful mindset!
The close date of the market is 2025-01-02T07:59:00Z, but in the description there is:
> Resolves YES if, at the end of 2025
Is it expected that the close date is about 1 year earlier than the planned resolution? (Probably this is expected, asking to be sure that there is no error is market close date or in the description)
@EliezerYudkowsky It's at least quite contentious in the open source community. Pre-existing large companies have to apply for a license and the terms forbid using the weights to train another model. There are also restrictions on using it to break the law, write a virus, or generate spam, and I'd think at least the last two disqualify it. It should count for this market anyway because LeCun is definitely including Llama in his claim. https://www.businessinsider.com/meta-llama-2-ai-model-not-open-source-2023-7?op=1
@Frogswap This suggests that a good definition of “open source model” would make the question clearer