AI Hallucination Nearly Triggers US Strike on Chinese Vessel — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Military aircraft were airborne this spring when officials realized AI-hallucinated intelligence was driving an armed operation against a Chinese vessel; the mission was aborted at the last minute, narrowly averting conflict with China, CNN reported Friday.
- The false intelligence report, which circulated during the war with Iran, claimed the target vessel was carrying components for a nuclear weapons program.
- A Special Operations Command analyst queried an AI chatbot to synthesize open-source data with classified signals intelligence; the chatbot misidentified the ship's cargo manifest, and the analyst then used the tool a second time to format the erroneous findings into an official-looking summary that spread across command channels.
- The Pentagon has framed AI as delivering a significant advantage in speeding up its kill chain, but the source notes the same speed 'may also allow hallucinations with insufficient human oversight' to travel up the chain before being questioned.
- Jake Steckler, a GovAI research scholar and veteran U.S. Army officer, said 'it's especially critical' for any decisions that could lead to use of force, and called the near-miss 'a call to add more safeguards to AI, not a reason to avoid it.'
Why it matters: A hallucinated cargo manifest traveled from one Special Operations analyst's chatbot query to aircraft already airborne — exposing how the Pentagon's race to compress its kill chain with AI can propagate unchecked errors into life-or-force decisions. Steckler's framing puts the burden on commanders to build verification into adoption before speed becomes doctrine.
Ask SkimNews
