Before 2032, will we see an event precipitated by AI malfunction that causes at least 1000 deaths and/or at least $200B 2021 USD in economic damage?

Resolves the same as the original on Metaculus.

Resolution criteria

This question resolves as Yes if, before January 1, 2032, there are three credible media reports indicating that there has been one or more events precipitated by AI malfunction which caused either 1000+ deaths or $200bn 2021 USD in economic damage.

Multiple incidents stemming from the same source can count for the question to resolve as Yes. To be considered from the same source the incidents must be attributable to a specific flaw or version of an AI. For example, deaths from a prior major version of Tesla’s self-driving car software would not be summed with deaths attributable to later versions.

To count as precipitated by AI malfunction an incident should involve an AI system behaving unexpectedly. An example could be if an AI system autonomously driving cars caused hundreds of deaths which would have been easily avoidable for human drivers, or if an AI system overseeing a hospital system took actions to cause patient deaths as a result of misinterpreting a goal to minimise bed usage.

To be considered easily avoidable an incident must not involve human negligence. For example, if a human is expected to monitor the AI and promptly intervene, a failure to intervene resulting in death or economic damage would not be considered attributable to an AI malfunction unless the AI malfunction was such that a human could not reasonably be expected to prevent it. If, for example, the Boeing MCAS system had been an AI system and there was no possibility for the pilots to override its decision to lower the aeroplane nose, leading to a fatal crash, this would count for resolution.

Deaths or damage caused by an AI's expected and intended behavior will not be included; for example, AIs used to target enemy combatants in war would be excluded, as well as AIs successfully performing a medical treatment as specified, despite the treatment causing harms that the doctors or designers were unaware of. An unexpected malfunction of an AI system used in war leading to collateral damage would qualify for the question to resolve as Yes.

Fine print and additional background information can be found on Metaculus.

Once the original resolves, its resolution will be applied to this market automatically. Trustworthy-ish users are encouraged to resolve this market before then if the outcome is known and unambiguous. Feel free to ping @jskf to request early resolution or to report issues.