This is an experiment with a Manifold version of the Change My View idea from Reddit. The idea is that someone comes with a certain view that they have, and commenters give their best arguments to change their mind. If you manage to change my view on the topic, I give you 200 Mana. You don't need to completely flip my view, any significant change on a specific aspect or a softening of my position is enough. I will add additional mana to the bounty as needed, as well as give higher rewards to larger changes in my view. The other rule is that top-level comments must challenge my view and argue in good faith.

Despite being a member of the EA community and working in AI, I am so far still unconvinced of AI existential risks, the need for AI alignment, etc. I think these concerns are mostly driven by hype and misunderstanding. A big part of that is simply that so far, I just haven't heard any strong argument for it, and I'm hoping this post will change that. Also, to clarify, I am not against AI ethics in general, just the futuristic, long-term version of AI alignment.

Here's why I don't find the arguments I heard so far convincing:

• A lot of the arguments seem to be based on the notion of "How do you know it will not happen? Can you really be sure?". I think that's a very weak argument that's basically equivalent to flipping the burden of proof, and I'm pretty sure the burden of proof lies with the AI risk believers here. I can go more in-depth into why this line of reasoning is unconvincing to me but let's leave it here for now.

• Some arguments, like the Rational Animations video about the subject, basically just say "experts are concerned". A lot of experts are also not concerned. People like Andrew Ng have come out saying that they think these concerns are overhyped, and I know a lot of people working in AI who would agree. I think certain actors like Elon Musk and Sam Altman also have financial motivations to fearmonger on this.

• Some arguments seem to be extrapolating some exponential growth into the future. There are very real constraints to improvement in AI that mean that exponential growth is very likely to be short-lived. We are approaching the limits in the available data and in the financially feasible amounts of compute. In fact, most of the recent improvements in AI have simply come from investing more and more money in training, which is obviously unsustainable. This means that further breakthroughs will need to come from improved algorithms. Despite an explosion of AI researchers, the research in the area is suffering from some amount of slowdown already.

• The last family of arguments seem to talk about AI empowerment and the rise of superintelligent AGI as some kind of inevitable process. I don't get why any of this is inevitable. Few things in the world are inevitable.

You can try to change my mind on any of these argument groups, but you'll probably have the most success if you give an argument that doesn't fall into these categories.

Here are some reasons I don't think AI existential risks are important:

• I don't think The Alignment Problem is as new and scary as some people believe. Our society is already filled with alignment problems. In economics, it's called the principal-agent problem, and in politics, it's called "Why are the politicians I voted for doing this shit?". We regularly deal with entities of incredible power, much greater than that of any of us, who have their own interests which sometimes are contrary to ours. These entities are states and corporations. I wouldn't say we have been very successful in dealing with them, but despite the many flaws, these entities make our world a better place than it would have been without them (sorry Anarchists)

• I think that Alignment and AI risks are too poorly defined and too far in the future to be tractable issues. How can we stop some future superintelligence when we have no idea what that would even mean? I think there might be a time in the future where it will become clearer what the actual risks are, and that allocating resources to this problem would make sense at that point in the future, not now.

• Another important aspect for cause prioritization is neglectedness. I don't think that AI alignment is neglected right now. It seems to me that it receives a lot of attention and funding. On the other hand, I'm concerned about what impact all this funding has brought. Have we come any closer to solving this issue?

• This bullet point is just vibes, but I feel like some people that are so concerned about AI that they somehow lost the line between science fiction and reality. They are zealous to the point where they just don't pass the sniff test. Whenever I see someone saying p(doom) > 80%, it seems like they're some part of a doomsday cult. When your view of reality is so far away from the mainstream, I think you need to very strongly question whether everyone else is crazy, or maybe it's just you. This isn't exactly a rational argument but I wanted to point this out, maybe to the benefit of those of you who are concerned with the movement's optics.