Will China be competitive in the LLM race compared to OpenAI, Anthropic, and Google by end of 2024?
Dec 31

To be judged as an LLM made by a Chinese organization that ties or surpasses the leading LLM by OpenAI, Anthropic, or Google on the leaderboard here.

Additional Information

China is significantly investing in the development of large language models (LLMs) and is home to many AI-oriented companies and LLM applications. Chinese tech giants like Baidu, Alibaba, Tencent, and SenseTime have already released their GAI products, and the Chinese government aims to be an AI leader by the 2030s. While there isn't direct information indicating whether China will surpass or match OpenAI, Anthropic, and Google by the end of 2024, given the substantial investments and efforts, it's plausible that China will remain competitive in the LLM race.

Concurrently, the progress and development of OpenAI, Anthropic, and Google in the LLM race have been rapid. OpenAI led the initial LLM boom with its GPT-3 model, Anthropic is focusing on making LLMs more transparent, safe, and beneficial, and Google's Pathways AI model has surpassed GPT-3 in terms of parameters. This suggests that the race in LLM development continues to be highly competitive.

Some Background From The Web

Will China be competitive in the LLM race compared to OpenAI, Anthropic, and Google by end of 2024?

What are China's current advancements and investments in the field of LLM?

What is the pace of progress and development of OpenAI, Anthropic, and Google in the LLM race?

Get Ṁ600 play money
Sort by:

@Ledger If a Chinese model surpasses Google's model, but not Anthropic's or OpenAI's, does this resolve YES?

Yi-XLarge is in the oven cooking right now.

@Sss19971997 Yi-Large already score 51% on Alpaca-Eval2

due to sheer numbers of engineers there are almost certainly CCP agents inside google and as of 11 years ago when I worked there almost all the source code is visible to almost all employees so it would be pretty easy for them to steal the code eh?

@JonathanRay tho the code is probably unusable outside google because they use their own special snowflake infrastructure for everything. A lot of it would have to be rewritten

@JonathanRay Please look into how much compute, experimental runs and cleaned data is needed, and how hard building a large language model at the scale of Opus, GPT4, Gemini is (it's a lot, and it's very hard). Nevertheless I believe that by EOTY 2024 best Chinese ML researchers can pull that off, their Qwen 1.5 line is quite good. It's not as simple as get training code -> train.

@MADGAMBLER6969 Stealing the code and having the government pay for a zillion GPUs would go a long way

@JonathanRay efficient training code is architecture specific (what interconnects you have, what makes the calculations), most models right now are on the neox/llama base and those codebases are open source, saying that China steals tech is generally true but there isn't anything to steal in LLMs, it seems like there are very few secrets in how to make a good model, it's just very difficult and needs scale (but scale alone is not enough). Zillion GPUs is also a fantasy because there are more important places for compute (that today costs premium because of the bans on imports of advanced silicon) to be allocated in China, still some labs (Alibaba and Sensetime are main ones) have compute to participate in that dickwaving contest and I think they will overcome Google (they have a pretty bad track record for being leaders in cutting edge tech).

opened a Ṁ100 NO at 50% order

are people pushing the market up because of sensetime?

@PaulHabermas what happened😱😱😱

sold Ṁ35 YES

I think the ambiguity is in "'by' EOY 2024". Does it mean resolving the market to YES if, at any point before 2024, a Chinese model surpasses one of the three? Or, it means resolve according to the situation on Dec 31?

@Sss19971997 @Ledger wrote "So as soon as Alibaba shows up in the organization column before the first appearance of OpenAI, Google, or Anthropic, then this will resolve yes."

"as soon as" implies that it resolves YES if a Chinese model surpasses one of the others at any point from now until EOY 24.

Qwen1.5 72B is higher than Gemini Pro


The criteria is that it surpasses the leading model by OpenAI, Google, or anthropic.

This is what I see on the leaderboard...

@Ledger Bard is not a "model"

@Ledger So technically Alibaba has surpassed google

@Sss19971997 well it's under the model column.

strictly speaking the leading google LLM is gemini ultra, and it's not on the leaderboard at all

@jacksonpolack the criteria for this is the leading LLM on the leaderboard. The leaderboard even lists the organization if you scroll to the right. So as soon as Alibaba shows up in the organization column before the first appearance of OpenAI, Google, or Anthropic, then this will resolve yes.

@Ledger could you clarify the meaning of "or" here? To resolve YES, must a Chinese LLM surpass the leading LLM from every company out of Google, OpenAI, and Anthropic, or is it enough to merely surpass the leading LLM from one of them?

@chrisjbillington or means any of the three companies.

@Ledger well if you consider models not on the board, then the best Chinese is GLM4, which should be better than claude 1. It will also resolve the market to yes in that sense.

@Sss19971997 It's not obvious when you're writing it, but your comment is also ambiguous.

"Must be higher than any of them" can mean "must be in first place" or can mean "must be higher than any one of them".

The comment I was replying to above makes it sound like it has to be in first place, but isn't definitive. And the market description is just ambiguous.

Unfortunately one must use more precise language for these sorts of things. When you have a particular interpretation in mind, ambiguity like this can be invisible, but it's very much there.

@chrisjbillington yea. The ambiguity is what stopped me from betting all the way to 100%. But I think or seems quite clear in logic, i.e., any of the three statements resolving True will lead to YES.

@chrisjbillington The antonym for "any" is "all"

@Sss19971997 not really, in some contexts they're synonyms.

If I claim that "I can beat any grandmaster at chess", the claim is that I can beat an arbitrary grandmaster, not merely one of my choosing.

But if I say "if I lose to any grandmaster, I'll hang my head in shame", that means I could win against all but one, and still have to hang my head in shame.

It's ambiguous.

Actually I think that second one is still a bit unclear tbh. If you want to communicate either of these things you should word it differently to avoid the ambiguity.

More related questions