As AI grows more capable of identifying software vulnerabilities, experts are increasingly warning of a potential disaster scenario: the so-called “Vulnpocalypse.” Hackers could quickly turbocharge their attacks with AI technology designed to identify holes in cyber defenses, security researchers warn. This week, that scenario started to feel less theoretical.

Subscribe to read this story ad-free

Get unlimited access to ad-free articles and exclusive content.

Anthropic, a leading AI company, announced that it would withhold its latest model, Mythos Preview, from the public, citing unprecedented vulnerability-discovery capabilities that could cause significant damage in the wrong hands. The company is instead sharing the model with a limited group of tech giants and partners to help shore up their defenses.

The concern has reached the highest levels of government. In the wake of Anthropic’s announcement about Mythos Preview, Treasury Secretary Scott Bessent convened a meeting with major financial institutions this week to discuss “the rapid developments taking place in AI,” an agency spokesperson said.

Some theorize that AI could help hackers crash financial systems or lock up hospitals and manufacturing plants. It could help countries like Iran shut down American critical infrastructure. Or it could be used to cause mass system outages affecting travelers or internet users.

“We have way more vulnerabilities than most people like to admit; fixing them all was already difficult, and now they are far more easy to exploit by a far broader variety of potential adversaries,” said Casey Ellis, the founder of Bugcrowd, a platform for cybersecurity researchers who hunt down vulnerabilities. “AI puts the kind of tools available to do this in the hands of far more people.”

Hackers often break into systems by figuring out ways to exploit flaws in software, leading to an endless back-and-forth where attackers will look for new opportunities and defenders try to update their code to block them. Some AI models, particularly ones that are as good or better as a person at coding, have proven to be extremely adept at rapidly discovering those vulnerabilities.

Worries about AI’s ability to give hackers a superweapon that overwhelms cybersecurity defenses hit a new high this week, when Anthropic announced that it would not yet release Mythos to the public.

But regardless of whether Mythos lives up to its hype, industry experts largely agree that a period of reckoning is likely coming soon, when hackers will be able to use AI to give them more of an advantage over their victims than ever before.

“A defender needs to be right all the time, whereas an attacker only needs to be right once,” Ellis said.

Logan Graham, who leads offensive cyber research at Anthropic, said that even if Mythos were never to become public, he expects the company’s competitors, including those in China, to release models with comparable hacking ability in the coming months and years.

“We should be planning for a world where, within six months to 12 months, capabilities like this could be broadly distributed or made broadly available, not just by companies in the United States,” Graham told NBC News.

“If you step back, that’s a pretty crazy time frame, where usually preparations for things like this take many years,” he said.

Mythos is not simply good at finding vulnerabilities, Graham said, but also at chaining them together into complicated exploits that can be devastating hacking tools.

Katie Moussouris, the CEO and co-founder of Luta Security, a company that connects vulnerability researchers with software developers, said she expects scenarios similar to when major cloud providers go offline with glitches and take significant chunks of the internet with them.

“We absolutely are going to start to see big outages that have downstream effects on other industries, like the airline industry suffered in the CrowdStrike incident. Various other things suffer when Cloudflare is down, when Amazon Web Services are down,” she said.

Cynthia Kaiser, a former senior cyber official for the FBI and a senior vice president at Halcyon, a company that works to prevent ransomware attacks, said she is concerned about how AI will help mediocre hackers whose only limitation from attacking hospitals to hold them for ransom is the fact that they lack the skill.

“The wannabes, this undercurrent of people who have not been capable of doing these operations just a year ago, now have some of the most powerful tools ever known to humankind in their hands,” she told NBC News. “Health care and critical manufacturing were the most targeted by ransomware attacks last year. I think that pattern would follow. They’re going to go after areas where there’s little tolerance for downtime.”

AI also could have significant impacts for cyber warfare and attacks on U.S. critical infrastructure by giving a leg up to hackers whose goal is simple destruction.

Since the U.S. war with Iran began, Tehran’s hackers have gone after multiple American targets but repeatedly exaggerated their capabilities. They have notched only a single significantly destructive public attack — on a Michigan medical technology company called Stryker.

Federal agencies said this week that Iran has had some success hacking into critical infrastructure companies, including water and wastewater services and the energy sector, with the intent of causing disruption. It’s unclear if any of the attacks have been significant, and the victims have not been publicly identified.

But AI could make that job easier. Some industrial control systems have significant cyber defenses, though others — some water treatment plants in sparsely populated areas of the country, for instance — do not. Such systems are often notoriously challenging for hackers because they rely on more obscure systems.

Jason Healey, a senior research scholar at Columbia University who specializes in cyber conflict, said that while Iran has so far been unable to conduct a sophisticated cyberattack on the U.S., AI could make one more feasible.

“Instead of having to train up a generation of hackers that understand water works, AI should be able to help understand those systems and automate the process of intrusion,” he said.

Bryson Bort, the founder of Scythe, a platform that helps industrial systems imagine potential cyberattacks, said that critical infrastructure is often cut off from the internet, making a true doomsday scenario unlikely.

“Not all of these things lead to immediate, like, everyone starts dying like we’re in a Hollywood movie,” he said.

But it’s feasible that persistent hackers with the right access could keep attacking systems like water treatment plants and force them to temporarily stop working until they could regain control, he said.

“If it keeps getting compromised, I do need it to work, to actually produce water at some point,” he said.