Topline
Anthropic banned accounts that possibly sought to use its Claude AI model to create biological weapons, the company announced Thursday, revealing findings from an investigation just days after AI experts sounded the alarm on the potential for increasingly powerful digital assistants to bring about human extinction.
Anthropic revealed the possible attempt in a report published Thursday.
Photo by Joel Saget / AFP via Getty Images
Key Facts
Anthropic said in a report Thursday it blocked a request for Claude’s assistance in authoring a grant application for scientific funding that would have gone toward genetic alteration research into the mosquito-borne chikungunya virus.
The grant sought to make the virus, which has a very low mortality rate but causes long-term joint and muscle pain, more harmful, according to Anthropic, while acknowledging similar research could be used in vaccine or therapeutic development for the virus.
However, the research could also be used to make the virus more dangerous, and Anthropic was inclined to believe the project was not as harmless as it appeared because although the grant application suggested involvement by civilian researchers, the research was planned to be carried out at a military research institute.
Anthropic has not disclosed the names or associations of the institutions or researchers possibly involved, adding it banned all associated accounts, took down relay networks that avoided its regional blocks and shared its findings with AI labs and the federal government.
Crucial Quote
“We infer from the existence of these research materials that the research effort was not limited to a funding proposal,” Anthropic said in its report. “We are banning accounts we detect associated with this cluster of activity and are continuing to incorporate our investigative findings into new measures to detect and prevent the actors from misusing Anthropic’s services.”
News Peg
A former OpenAI and Anthropic staffer, as well as an Anthropic employee, said this week there is a possibility AI could cause human extinction within the next decade. Evan Hubinger, who oversees efforts at Anthropic to align AI functions with human goals and values, said he personally believes there is a more than 10% chance “AI could kill all humans” in the next 10 years if development continues at its rapidly accelerating pace.
This is a developing story. Check back for updates.
Leave a comment