Skip to main content
MANIFOLD
Are reports that a USAF AI drone went rogue in a simulation substantively accurate?
8
Ṁ670Ṁ1.1k
resolved Jun 2
Resolved
NO

https://www.vice.com/en/article/4a33gj/ai-controlled-drone-goes-rogue-kills-human-operator-in-usaf-simulated-test

Resolves yes if further credible reporting (that doesn't just reference the original blog post) that :
- US military was using ML to train a vehicle to target enemy weaponry in a simulation,
- the system learned to target friendly operators or comms, consistent with maximizing its reward through avoiding human 'no-gos' or interference that can stop it taking out enemy threats, a la 'reward misspecification', &
- this was unintended (i.e. this specific failure mode was not intended in advance as, for example, a reward misspecification proof of concept)

Resolves no if credible reporting that this wasn't the case.


Will otherwise resolve EOY 2023, likely to market if there is no further evidence either way. I won't bet myself.

Market context
Get
Ṁ1,000
to start trading!

🏅 Top traders

#TraderTotal profit
1Ṁ175
2Ṁ150
3Ṁ45
4Ṁ13
5Ṁ3
Sort by:

Had it not occurred to the OP that when the Air Force now says they're not simulation-training killbots to sometimes kill without authorization, they might be lying?
Like, "The Department of the Air Force has not conducted any such AI-drone simulations" is not actually easy to believe? It depends what they meant by "such simulations", but this sort of thing is not far beyond what would go on in the kind of stuff we'd expect them to be doing/know that they're doing.

@makoyass Seems like a possibility, but one that's tough to falsify! I was happy to resolve given (a) the origin of the story himself, through the organization that originally quoted him, clarified his statements in a way consistent with this resolving negative (b) Edouardo seems trustworthy and also claimed it was just a scenario



I'll pay out Duncn if it turns out I jumped the gun :)

Hmm what's your P(Edouard would go along with an order to trash the narrative even though he's not part of the DAF because he believes it will reduce race dynamics)?
But never mind, don't answer that question, if he made that decision, I don't want to talk about it either.

predictedNO

AI-Controlled Drone Goes Rogue, 'Kills' Human Operator in USAF Simulated Test (vice.com)

"After this story was first published, an Air Force spokesperson told Insider that the Air Force has not conducted such a test, and that the Air Force official’s comments were taken out of context."

""The Department of the Air Force has not conducted any such AI-drone simulations and remains committed to ethical and responsible use of AI technology," Air Force spokesperson Ann Stefanek told Insider. "It appears the colonel's comments were taken out of context and were meant to be anecdotal.""

Even without the individual claiming to personally know the situation was made up, it is crazy to estimate more than 10% chance this actually happened.

🤔