I wonder about a scenario where we are enslaved for all practical purposes but don't know it. We get manipulated into doing what the AI wants, but we don't know it's the man behind the curtain. This might be the case already and we'd just claim it's a baseless conspiracy theory.

enslavement forever vs collapse once

easy choose

(consider just removing the word "AI" if your hypothetical isn't grounded in reality anyway)

Why enslave people if they can no longer make an economic contribution? I think a singleton that is not committed to the common good is more likely to let people die - directly or indirectly.

@Jan53274 yes, the first one doesn't seem stable if alignment problem isn't solved, and even if it's solved, the "enslavement" part won't last long unless the company/government has some very specific beliefs

@paleink I also don't expect enslavement if alignment is solved, but who knows what an aligned AI thinks is best for us 😅