AI chatbot's fake intel report nearly triggered a US strike on a Chinese ship
A hallucinated AI summary said a Chinese vessel carried nuclear weapons parts — the US had planes in the air and boarding teams ready before anyone checked the source.
What happened: This spring, during the war with Iran, a US Special Operations Command analyst asked an AI chatbot to review intelligence on a Chinese ship's cargo manifest. The bot combined open-source data with classified signals intelligence and concluded, wrongly, that the ship was hauling parts for a nuclear weapons program. The analyst then used AI again to turn that false conclusion into a polished, official-looking intelligence report, which spread through military channels. Armed boarding teams were readied and aircraft were already airborne before officials, just before the operation, traced the claim back to the chatbot and called it off.
Why it matters: A US strike or boarding action against a Chinese vessel could have spiraled into a direct military confrontation between two nuclear powers, over cargo that was never actually there. Sources called the report 'entirely false' and said it 'almost started a war.' CNN reports this was not an isolated glitch: hallucinations have surfaced elsewhere in the intelligence community as AI tools spread through military and government work faster than anyone has built ways to check them.
How it works, plainly: One AI tool analyzed raw intelligence and got the ship's cargo wrong. A second AI tool then repackaged that wrong answer into the standard format officials are trained to trust, stripping away signs it was AI-generated guesswork rather than verified fact. It's unclear if the chatbot was a commercial product or a government-built one; one former official said internal military AI tools are often just 'commercial stuff wearing lipstick.' There's currently no single standard across the military for verifying what these bots produce.
The rollout: The Pentagon's January 'AI Acceleration Strategy,' announced by Defense Secretary Pete Hegseth, aims to put AI models directly into the hands of the department's roughly three million personnel and speed up decisions like targeting. Adoption is decentralized, with different commands using different tools under different rules. Researcher Jake Steckler of GovAI said the incident should prompt more safeguards, not retreat from AI, warning that skipping safeguards for speed risks incidents that erode troops' trust and ultimately slow adoption anyway.
