From https://metaculus.com/questions/17112/cybersecurity-laws-for-powerful-ai-by-2030/

Many AI experts worry about powerful AI getting into the hands of bad actors—those who might use such powerful AI to cause damage or for personal gain that inflicts costs on others—via hacking. A bad actor might be a black hat hacker, a terrorist group, or a rogue state.

One way to decrease the chance of this type of hacking succeeding is for those building powerful AI, the leading AI labs, to have strong cybersecurity measures in place. Given how high the stakes might be (see, e.g., “AI risk”), policymakers may decide to not bank on AI labs attaining strong enough cybersecurity by default.