@ScottLawrence Aggressive semantic filtering of chatbot output (as just one example way they might move forward) wouldn't require solving the alignment problem; the solution doesn't need to be perfect.

@ScottLawrence Why do they need to solve the alignment problem? They just need automated detection of censorable text. This seems like a thing that China is very, very, very good at, and has plenty of resources to bring to bear on.

@Boklam and @ML Took me a long time to come up with a vaguely coherent answer to this. And I apologize, because this still isn't very good. I have a strong intuition that censorship-heavy regimes will be very uncomfortable about public use of LLMs, and I've had a lot of trouble making the "why" concrete.

I'm not quite sure why, but people apparently want AIs to be superhumanly politically correct. In the U.S., we're okay with tech companies not censoring people who say "inappropriate" things in private chat (or even on their blogs), but it's somehow absolutely essential that the GPTs not behave that way.

I have very little knowledge of how censorship works in China, but my sense is that there are many ways of discussing censored topics while not being censored (some of which require knowledge of Chinese to understand, others of which might be described as "being Straussian"---in the West you can often get away with making many claims just by couching them in scholarly language, and I imagine the same is true in China). Censorship of humans doesn't need to be perfect, and so doesn't catch that stuff. Censorship of an AI... again, I don't understand why this is the case, but it seems the bar is much higher.

As far as whether this task is alignment-complete. Well, it depends on how much perfection is required. In the extreme case of "AI not allowed to break censorship rules even by being Straussian", yeah, I think this is alignment-complete. The reduction is simple: you can just add "don't say things that might cause the destruction of humanity" to the censorship rules. If you're okay with a Straussian AI then that's different, but I expect China's censors to not be happy with that.

Somewhat related: although I'm surprised by the high probability on this market, I'm also very happy about it. I expect what counts as "appropriate" in China will differ dramatically from what counts in the U.S., and I look forward to having access (even if through a translator) to a chatbot with very different training data.