Did we learn nothing from sci-fi? A study published in Science (via The New York Times) details how researchers used an AI model to design new viruses capable of infecting and reproducing inside bacteria. The good news: they took appropriate precautions, and their work could lead to breakthroughs in gene therapy and biotechnology. The bad news: the next guys might not be so generous.

Scientists at Palo Alto’s Arc Institute and Stanford University created the viruses with the genome language models Evo 1 and Evo 2. They work somewhat like large language models (LLMs) such as ChatGPT or Claude. Except instead of being trained on books and websites, they studied genetic sequences. The project trained its models on trillions of nucleotides (the building blocks of DNA) and learned the “grammar” of genetic code.

For this experiment, the team further trained the models on about 15,000 viruses from the same family as Phi X-174, a tiny virus that only infects E. coli bacteria. The scientists explicitly limited the scope to exclude anything that could infect humans, other animals, plants or fungi.

Evo generated around 700,000 possible new viruses. The team then selected the 285 most promising candidates, manufactured their DNA and inserted it into bacteria. In the end, 16 of Evo’s creations produced working viruses. They were at least as viable as Phi X-174, with some reproducing faster.

The story is the same as with many AI tools — or nuclear energy. Models like Evo could do enormous good in the right hands or enormous harm in the wrong ones.

AI-designed viruses could improve gene therapies, help researchers better understand genomes and produce better tools for attacking harmful bacteria. In a tightly regulated environment with ethical guardrails, this could be the kind of breakthrough that makes the world a better place.

But in our current regulatory environment, well, let’s just say we’re not quite there. Researchers have already shown that standard chatbots can provide some, shall we say, concerning help with planning biological weapons. A model that can design genomes takes that threat to another level, with more advanced versions potentially helping someone design a highly contagious or deadly virus. Fun times.

The science itself isn’t inherently evil. But when AI can move from describing biological threats to designing them, it isn’t enough to sit back and hope that only “the good guys” use it, as appears to be the case here. With the technology advancing at breakneck speed, will our guardrails catch up before it’s too late?