From https://metaculus.com/questions/16014/ai-unauthorized-access-before-2033/

The potential capabilities of artificial intelligence may radically shift our society. This could be in positive or negative ways – including extinction risk.

Because of this, it’s important to track the development of goal-oriented independent thought and action within AI systems. Actions that might not have been predicted by their human creators and that are typically seen as morally wrong are particularly interesting from a risk perspective.

Machine learning driven systems are already being used to aid human hackers in their efforts. For instance, see this list of cases, including one in 2018 where an AI controlled a bot net in a DDOS attempt that allowed a hacker access to Task Rabbit’s data for over 3 million users. However, up to now, we’ve had no reports where an AI has taken a similar action independently (ie. without human guidance).