I've alluded to/mentioned it in a couple markets before, but I have DID (or OSDD-1b, not sure which) and a "fun" side effect of that is that there's approximately ten alters/self-states in this one brain. Usually we're all basically in agreement on major decisions - it makes sense, we share a neurological substrate - but sometimes, one or two alters can cause some pretty significant issues for the system as a whole. One of us was talking to the others about a certain alter of ours who has a tendency to make questionable decisions, and after stumbling on their words for a bit, decided to call that alter "poorly aligned." That got me thinking.
A: I should probably touch more grass.
B: There's some interesting similarities between alters in DID/OSDD-1b and hypothetical AGI.
Both are currently poorly understood, with research ongoing. Until recently, a lot of psychologists didn't take DID seriously, and a lot of computer scientists didn't take the progress towards AGI seriously. It's not known how alters form, nor what predisposes them to have certain traits, but it is known that they exist and can have measurable effects on the body, particularly in the nervous system and cardiovascular system. Similarly, we don't know how an AGI would form nor what traits it would have, but we can say with pretty high certainty that the existence of an AGI is at least physically possible.
Both can have alignment problems. "Misaligned" alters (typically called "persecutors") can harm the system by making poor choices, alienating people, or self-harming in ways that affect the entire system. A misaligned AGI could theoretically end the world (though I personally think it's more likely that a hostile AGI would use manipulation to engineer a slow, inevitable takeover. Frog in boiling water and all.)
Both respond poorly to ostracization and othering. I think this is why I sometimes bristle at the way people talk about AGI - like alters, AGI would be sentient. Threatening to switch off an AGI's power supply is a death threat. Much like alters in a system, who must all exist as equals if the system is going to function, AGI must be treated as equal, sentient beings. Alters who are ostracized, othered, and excluded become more resentful, not less. I think the same is going to be true of AGI - if we spend all our time hyping it up as a threat and saying "it's fine, if something goes wrong we'll just kill it," it's less likely to trust us. Sentient beings always ought to be treated as such.
Before anyone asks - no, we don't use alts. We share a single account, i.e. this one. Even a single alt would be too much for us to handle - let alone the several we'd need if each alter wanted to make their own. Like basically anything this system does, this account is a collaborative effort.