BP
A month ago, Elizabeth Sherwood-Randall laid out a scenario in Foreign Affairs. She is not a science fiction author. She was White House Homeland Security Adviser from 2021 to 2025. She coordinated the response to the 2024 H5N1 (Bird Flu) outbreak in US dairy.
She wrote:
In the foreseeable future, a terrorist group with minimal skills could access a jailbroken AI model… The group could use the model to design a new strain of H5N1 avian influenza that is more lethal and more easily spread from animals to humans than existing strains. Connecting remotely to one of the new generation of fully equipped, contract-for-service “cloud” labs, it could anonymously arrange for the virus to be built and tested and then sent to a biomanufacturing facility to be produced at scale. From there, the group could stealthily disperse the new strain through air-handling systems at major livestock farms across the United States.
Before long, the American poultry, beef, and dairy industries would suffer massive losses. When the virus inevitably jumped to the human population, it would cause severe illness and could result in hundreds of thousands or even millions of deaths.
The response from U.S. public and private institutions—lacking both an integrated strategy and adequate tools—would be too slow, too uncoordinated, too underresourced, and too overwhelmed to stop the spread and mitigate the consequences. And soon, the virus would also cross borders, triggering food insecurity, widespread sickness, and mass casualties around the world.
That same day, the Belfer report she co-authored, Deterrence by Resilience, made policy recommendations responding to this, including restricting unlicensed models (AI) and unscreened print shops (biomanufacturing).
The scenario may not be that imaginative.
On 10 September 2026, Anthropic published Detecting and Countering Misuse of AI, documenting cases of people using Claude to help with questionable purposes, from assembling weaponized drones to performing gain-of-function work on H5:
Here, we present five case studies of actors using our models in ways that could support biological weapons development… In the second, a researcher in an unsupported region spent weeks planning avian influenza mammalian-adaptation experiments with Claude, but classifiers confined the work to our weakest models…
The plan involved genetic-engineering approaches aimed at introducing mutations associated with mammalian adaptation and airborne transmissibility in animal models.
They were careful to add a caveat—in the preamble, no less:
The individuals implicated in these case studies are working scientists. We do not assert that they intended harm.
Sherwood-Randall is not the first to imagine such a crisis:
- February 2024 (Munich Security Conference): CEPI hosts a tabletop exercise entitled “Fighting ‘Smart’ Pandemics” wherein a group of senior people from health, security, and tech were presented with a scenario wherein a pathogen that appears zoonotic spreads rapidly around the world. An anarchist cell then claims they made it with AI.
- April 2026: RAND hosts a tabletop exercise 10 times, in which participants faced “a global pandemic triggered by students’ accidental release of an AI-engineered virus.”
The scenarios differed slightly across these exercises, but the policy recommendations mirrored those in the Belfer report: limit access to the technology that can design (AI) and print (biomanufacturing) a pathogen, and create biosurveillance to flag an outbreak as engineered.
Companies like Twist Bioscience and Integrated DNA Technologies, among others, accept a customer’s description of a gene sequence and synthesize it. Both claim to screen the customer and the sequence before shipping an order of DNA, checking each gene against a database of pathogens.
There are also benchtop DNA synthesizers available for a mere $300,000 that can print short DNA without going through a screened vendor.
No comments:
Post a Comment