are people pushing the market up because of sensetime?

@PaulHabermas what happened😱😱😱

I think the ambiguity is in "'by' EOY 2024". Does it mean resolving the market to YES if, at any point before 2024, a Chinese model surpasses one of the three? Or, it means resolve according to the situation on Dec 31?

@Sss19971997 @Ledger wrote "So as soon as Alibaba shows up in the organization column before the first appearance of OpenAI, Google, or Anthropic, then this will resolve yes."

"as soon as" implies that it resolves YES if a Chinese model surpasses one of the others at any point from now until EOY 24.

Qwen1.5 72B is higher than Gemini Pro

@Sss19971997

The criteria is that it surpasses the leading model by OpenAI, Google, or anthropic.

This is what I see on the leaderboard...

@Ledger Bard is not a "model"

@Ledger So technically Alibaba has surpassed google

@Sss19971997 well it's under the model column.

strictly speaking the leading google LLM is gemini ultra, and it's not on the leaderboard at all

@jacksonpolack the criteria for this is the leading LLM on the leaderboard. The leaderboard even lists the organization if you scroll to the right. So as soon as Alibaba shows up in the organization column before the first appearance of OpenAI, Google, or Anthropic, then this will resolve yes.

@Ledger could you clarify the meaning of "or" here? To resolve YES, must a Chinese LLM surpass the leading LLM from every company out of Google, OpenAI, and Anthropic, or is it enough to merely surpass the leading LLM from one of them?

@chrisjbillington or means any of the three companies.

@Ledger well if you consider models not on the board, then the best Chinese is GLM4, which should be better than claude 1. It will also resolve the market to yes in that sense.

@Sss19971997 It's not obvious when you're writing it, but your comment is also ambiguous.

"Must be higher than any of them" can mean "must be in first place" or can mean "must be higher than any one of them".

The comment I was replying to above makes it sound like it has to be in first place, but isn't definitive. And the market description is just ambiguous.

Unfortunately one must use more precise language for these sorts of things. When you have a particular interpretation in mind, ambiguity like this can be invisible, but it's very much there.

@chrisjbillington yea. The ambiguity is what stopped me from betting all the way to 100%. But I think or seems quite clear in logic, i.e., any of the three statements resolving True will lead to YES.

@chrisjbillington The antonym for "any" is "all"

@Sss19971997 not really, in some contexts they're synonyms.

If I claim that "I can beat any grandmaster at chess", the claim is that I can beat an arbitrary grandmaster, not merely one of my choosing.

But if I say "if I lose to any grandmaster, I'll hang my head in shame", that means I could win against all but one, and still have to hang my head in shame.

It's ambiguous.

Actually I think that second one is still a bit unclear tbh. If you want to communicate either of these things you should word it differently to avoid the ambiguity.

Since Gemini Ultra is not on the board yet, if some chinese model surpass gemini pro right now, then it resolve to yes, right?

@HanchiSun I am very confident that GLM 4 is on par with Claude thus better than Gemini Pro. It now depends on if Lmsys puts it on the leaderboard before Gemini Ultra

@Sss19971997 Bard does not count, right? since it is not just an LLM

Anyone care to disagree? Yes orders up to 10%

For how long must it surpass the others? Just an instant?

@Ledger does the "or" mean they have to have a better LLM than one of Anthropic, Google, or OpenAI, or all of?

@RobertCousineau , it would have to be better than the best performing LLM from any of the above mentioned, but not all.