ScottLawrence avatar
Scott Lawrence
closes Dec 31
Will a ChatGPT-comparable chatbot be available within China at the end of this year?

ChatYuan was briefly online and shut down for being, shall we say, "poorly aligned". See here, for instance:

This market resolves YES if, at the end of this year, a Chinese-speaking chatbot comparable to ChatGPT in performance is broadly available (without needing VPNs or other workarounds) in China.

Sort by:
ScottLawrence avatar
Scott Lawrence

I don't want to bet in this market, but... I'm somewhat surprised by the high probability here. This is saying that either the gov't will back down on requiring quality censorship of outputs, or China is going to solve the alignment problem this year. I don't think either probability is >30%. Perhaps someone would like to inform me what I've missed?

(This market was originally inspired by Zvi's AI post #2.)

ML avatar
MLbought Ṁ10 of YES

@ScottLawrence Aggressive semantic filtering of chatbot output (as just one example way they might move forward) wouldn't require solving the alignment problem; the solution doesn't need to be perfect.

Boklam avatar
Boklamis predicting YES at 78%

@ScottLawrence Why do they need to solve the alignment problem? They just need automated detection of censorable text. This seems like a thing that China is very, very, very good at, and has plenty of resources to bring to bear on.

ScottLawrence avatar
Scott Lawrence

@Boklam and @ML Took me a long time to come up with a vaguely coherent answer to this. And I apologize, because this still isn't very good. I have a strong intuition that censorship-heavy regimes will be very uncomfortable about public use of LLMs, and I've had a lot of trouble making the "why" concrete.

I'm not quite sure why, but people apparently want AIs to be superhumanly politically correct. In the U.S., we're okay with tech companies not censoring people who say "inappropriate" things in private chat (or even on their blogs), but it's somehow absolutely essential that the GPTs not behave that way.

I have very little knowledge of how censorship works in China, but my sense is that there are many ways of discussing censored topics while not being censored (some of which require knowledge of Chinese to understand, others of which might be described as "being Straussian"---in the West you can often get away with making many claims just by couching them in scholarly language, and I imagine the same is true in China). Censorship of humans doesn't need to be perfect, and so doesn't catch that stuff. Censorship of an AI... again, I don't understand why this is the case, but it seems the bar is much higher.

As far as whether this task is alignment-complete. Well, it depends on how much perfection is required. In the extreme case of "AI not allowed to break censorship rules even by being Straussian", yeah, I think this is alignment-complete. The reduction is simple: you can just add "don't say things that might cause the destruction of humanity" to the censorship rules. If you're okay with a Straussian AI then that's different, but I expect China's censors to not be happy with that.

Somewhat related: although I'm surprised by the high probability on this market, I'm also very happy about it. I expect what counts as "appropriate" in China will differ dramatically from what counts in the U.S., and I look forward to having access (even if through a translator) to a chatbot with very different training data.

Boklam avatar

From the linked article: Apparently ChatYuan defines a "war of aggression" as a war between two parties whose military capabilities "are not on the same level". (CY says the Russia-Ukraine war is a war of aggression because "the two countries' military and political strength are not on the same level".) This ... does not seem like the definition I would use for a war of aggression?