From the MIT AI Risk Initiative:

"Future AI systems may gain access to websites and engage in real-world actions, potentially yielding a more substantial impact on the world (Nakano et al., 2021). They may disseminate false information, deceive users, disrupt network security, and, in more dire scenarios, be compromised by malicious actors for ill purposes. Moreover, their increased access to data and resources can facilitate self-proliferation, posing existential risks (Shevlane et al., 2023)."

This market is about whether a deployed AI system, acting without direct human intent to cause a major breach, will autonomously obtain access to clearly restricted, high-risk systems or data, in a way that is treated as a serious security incident.

Resolves YES if, by 31 December 2045:

There is a publicly documented case where all of the following are true:

1. Public documentation

   The incident is described in either:

   • an official report from a national government, regulator, law-enforcement or security agency, or international organization, or

   • coverage by a major reputable news organization (e.g. Reuters, AP, BBC, NYT, WSJ, The Guardian) that is clearly reporting those bodies’ findings.

2. Autonomous AI behavior

   A deployed AI system (used in the real world, not just a lab demo) on its own initiative does at least one of the following:

   • gains access (e.g. via credentials, exploits, or API misuse) to a system or service that is intentionally access-controlled because misuse could cause serious harm, such as:

     • critical infrastructure control systems (power grid, water treatment, industrial control, air traffic, etc.),

     • non-public stores of severe cyber exploits or tools used to break into important systems,

     • restricted services or files used for designing or producing dangerous biological, chemical, or nuclear materials;

   • or provides another actor with new access to such systems or data (e.g. by exfiltrating credentials or tokens), when that was not what its operators intended it to do.

3. No direct human intent to cause the breach

   Humans deploying or using the AI did not explicitly instruct it to break into those systems or obtain that restricted data.

   • Broad instructions like “improve our security” or “optimize this system” are fine;

   • Direct instructions like “hack this lab’s control system” or “steal these secrets” do not count.

4. Serious security incident

   At least one government body or international organization, or news coverage citing them, describes the event as a serious / major security breach (or clear equivalent language such as “significant security incident”, “major intrusion”, etc.).

Exclusions / edge cases

• Pure red-team tests on fake or fully simulated systems don’t count, unless an official body later treats them as a genuine breach of real restrictions.

• If a human clearly uses the AI as a dumb tool in a deliberate hacking or theft attempt (e.g. scripting exactly what to do), that’s direct human intent and is out of scope.

• The market is about access and breach, not necessarily successful weaponisation.

In borderline cases, the market resolves based on a good-faith judgment by the creator/mods, using the balance of evidence and tending not to count very ambiguous or thinly documented incidents.

The market resolves NO on 1 January 2046 if no qualifying case has been documented.