Anthropic says scientists used its artificial intelligence models in research that could potentially aid the development of biological weapons, prompting the company to tighten its safeguards.
The AI company in a report said it had disrupted several potential plots this year after detecting users attempting to use its models for advanced biological research.
Anthropic said it could not establish whether the research was actually intended for weapons development. That distinction is particularly difficult in biology, where the same research can have legitimate medical applications, such as in the case of vaccine development, while also potentially making pathogens more dangerous.
The company said it therefore chose to err on the side of caution.
Scientist Sought Help With Chikungunya Research
One of the cases highlighted by Anthropic dates back to May, when a scientist used Claude while preparing a grant application for research involving chikungunya, a mosquito-borne virus that can cause severe pain and other symptoms lasting for months.
The proposed work involved gain-of-function research, in which scientists alter an organism or virus to study how changes affect its behaviour. Such research can contribute to understanding diseases and developing vaccines, but it can also raise serious concerns when mutations could make a pathogen more harmful.
According to Anthropic, the researcher wanted to develop mutations that would increase the virus’s harmful effects as it was repeatedly passed through live animals.
The company said the case became particularly concerning because it could determine that the research was intended to take place at a military research institute.
Anthropic did not identify the scientist, institution, country or specific biological agents involved.
Researchers Allegedly Tried To Evade Safeguards
Anthropic said the researchers involved in the biological cases took steps to conceal what they were doing and bypass restrictions placed on its AI systems.
The scientists worked to “obfuscate the purpose of their research to evade our safeguards,” the company said.
Anthropic subsequently banned the accounts linked to the activity.
The company said tests conducted last year using older versions of its models showed that they were not capable of providing meaningful assistance for dangerous biological research. Its newer systems, however, can handle substantially more complex scientific work.
That increase in capability led the company to introduce stricter controls covering a broader range of biological queries with potential dual-use applications.
Weapons Design And Propaganda Among Other Abuses
The biological cases were part of a wider report from Anthropic detailing how its AI systems, including Claude, were misused over an eight-month period.
The company said it had also uncovered attempts to use Claude for conventional weapons development. The cases involved software related to firearms, missiles, armed drones and bombs, with six incidents linked to users in China, Russia and Yemen.
Anthropic did not name the Yemeni group involved, but the circumstances described in the report point to the Iran-backed Houthi militia. The use of AI by terrorist organisations has become an increasing concern for US security officials.
The company also identified suspected government-linked activity involving Chinese and Iranian actors, including efforts targeting dissidents and expat communities for surveillance.
In another case, Russian state media allegedly used Claude to produce online propaganda designed to appear as independent journalism. The material included fabricated claims about an election in Moldova.
Among the different forms of abuse identified by Anthropic, biological applications stand out because of the potential scale of the consequences.
