A false, partially AI-generated intelligence report nearly prompted the US military to attack a Chinese ship in the Middle East, CNN reports. The incident “almost started a war,” according to a source cited in the report, and highlights the risks of incorporating AI into the sensitive work of the US Department of Defense.

Engadget has asked the Department of Defense to comment on CNN‘s reporting. We’ll update this article if we hear back.

The assessment was put together by a special operations command analyst who asked an AI chatbot about the Chinese ship’s manifest. According to CNN, the chatbot combined “open-source intelligence with secret signals intelligence” to conclude that the ship contained the “components of a nuclear weapons program” — a claim that nearly led the military to apprehend the ship until officials found out it was AI-generated.

CNN wasn’t able to find out what the ship was actually carrying or what AI tool, whether commercial or designed for the US government, the analyst used. Given the Department of Defense’s push to adopt AI overall, there are potentially several options.

In January, Secretary of Defense Pete Hegseth published a memo directing the DoD to become “AI-first” and experiment with models from leading US AI companies. That didn’t work out quite as planned with Anthropic, who refused to let the military use its models for the development of autonomous weapons, and was briefly banned by the government, but it has led to several other AI partnerships. For example, SpaceXAI reportedly struck a deal to let the military use Grok in February, and NVIDIA, Microsoft and Amazon entered into a partnership to offer their AI technologies to the DoD in May. These kinds of government deals are lucrative, which is why companies pursue them. But helping spread AI adoption clearly carries risks that normal enterprise software doesn’t.