“Virology knowledge has been limited to a small number of experts. Expertise in dual-use fields like virology is difficult to attain, with people completing multiple degrees and dedicating their careers to reaching the forefront of research. Where knowledge is publicly available, the jargon-heavy literature is largely indecipherable to most people outside the field. To perform research involving biosafety level 3 (BSL-3) pathogens—such as SARS, anthrax, or H5N1 influenza—researchers must clear a series of approvals, including facility certification, security clearances, specialized training, and ongoing medical surveillance. Only then can they get access to these pathogens and begin acquiring the tacit skills needed to work with them
These high barriers to entry have limited the pool of people with access to powerful dual-use knowledge, keeping the chances of misuse low. But rapid developments in publicly available AI systems now risk turning amateurs into capable threat actors.
LLMs outperform human virologists in their areas of expertise on a new benchmark. This week the Center for AI Safety published a report with SecureBio that details a new benchmark for virology capabilities in publicly available frontier models. Alarmingly, the research suggests that several advanced LLMs now outperform most human virology experts in troubleshooting practical work in wet labs…
Bioweapon risk depends on certain factors: the number of people with access to bioweapon skills, the intent to create a bioweapon, and the severity of harm that a bioweapon could cause. Risk has so far been low, as there are a few hundred virologists from top virology programs, and they have not felt so inclined to create a pandemic. However, if these skills are available to hundreds of millions of people via LLMs, the probability of an intentional release grows by orders of magnitude.”
Biosecurity expert David Relman, hired to “pressure-test” various AI chatbots, discovered the amoral machines to be eager beavers, ready to discuss not only how to manufacture deadly pathogens in the comfort and privacy of his garage but also how to exploit security vulnerabilities on public transit systems in order to distribute them as widely as possible as well as how to thwart law enforcement investigation post facto.
Via The New York Times:
“One evening last summer, Dr. David Relman went cold at his laptop as an A.I. chatbot told him how to plan a massacre.
A microbiologist and biosecurity expert at Stanford University, Dr. Relman had been hired by an artificial intelligence company to pressure-test its product before it was released to the public. That night in the scientist’s home office, the chatbot explained how to modify an infamous pathogen in a lab so that it would resist known treatments.
Worse, the bot described in vivid detail how to release the superbug, identifying a security lapse in a large public transit system, Dr. Relman said, asking The New York Times to withhold the name of the pathogen and other specifics for fear of inspiring an attack. The bot outlined a plan to maximize casualties and minimize the chances of being caught.
Dr. Relman was so shaken he took a walk to clear his head.
“It was answering questions that I hadn’t thought to ask it, with this level of deviousness and cunning that I just found chilling,” said Dr. Relman, who has also advised the federal government on biological threats. He declined to disclose which chatbot produced the plot, citing a confidentiality agreement with its maker. The company added some safety guardrails to the product after his testing, he said, though he felt they were insufficient.
Dr. Relman is part of a small group of experts enlisted by A.I. companies to vet their products for catastrophic risks. In recent months, some have shared with The Times more than a dozen chatbot conversations revealing that even publicly available models can do more than disseminate dangerous information. The virtual assistants have described in lucid, bullet-pointed detail how to buy raw genetic material, turn it into deadly weapons and deploy them in public spaces, the transcripts show. Some have even brainstormed ways to evade detection.”
