Does it count if their datacenter burns down with workers inside it due to it overheating while on full-tilt training an AI, or does it have to be an action by the AI's output?

Maybe an LLM gives inaccurate information such as safety information, which someone is then influenced by, causing a physical disaster such as a fire.

Hard to see how OpenAI & co could do it, government systems seem like more of a fair bet, or systems in hospitals.

We are talking 100 die at once, like an aircraft crash? Rather than for example AI giving unsound medical advice occasionally causing harm, which is eventually estimated to have caused at least 100 deaths over lots of time.

@AlanTennant I'm delegating that decision to whoever resolves the market "Does an AI disaster kill at least 100 people before 2040?".

@PeterMcCluskey ok