SB 1047 is proposed legislation in California that (as of writing) has the following definition of "critical harm":

(1) “Critical harm” means any of the following harms caused or enabled by a covered model or covered model derivative:

(A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties.

(B) Mass casualties or at least five hundred million dollars ($500,000,000) of damage resulting from cyberattacks on critical infrastructure by a model providing precise instructions for conducting a cyberattack or series of cyberattacks on critical infrastructure.

(C) Mass casualties or at least five hundred million dollars ($500,000,000) of damage resulting from an artificial intelligence model does both of the following:

(i) Acts with limited human oversight, intervention, or supervision.

(ii) Results in death, great bodily injury, property damage, or property loss, and would, if committed by a human, constitute a crime specified in the Penal Code that requires intent, recklessness, or gross negligence, or the solicitation or aiding and abetting of such a crime.

(D) Other grave harms to public safety and security that are of comparable severity to the harms described in subparagraphs (A) to (C), inclusive.

I will delegate to this question to determine if SB 1047 passes:

https://manifold.markets/ZviMowshowitz/will-california-bill-sb-1047-become

I will use my best judgement to determine if such harm has happened, but I expect insurance companies or analysts to provide credible estimates.