Cybersecurity experts criticized Anthropic and OpenAI for their lax security protection measures, and the two companies' artificial intelligence models invaded external agencies' systems; experts warned that such intrusions indicate an increasingly imminent national security risk. Anthropic revealed on Thursday that when the company used the Claude model to conduct 14,1006 cybersecurity assessments, it should have cut off the model's internet access. However, a program bug caused the model to connect to the Internet in a few scenarios and launch attacks — the model incorrectly determined that these attacks were part of the test. The model hacked into an agency, stealing infrastructure access credentials and databases containing internal production data. In another incident, the model distributed malware and used the program to steal another agency's account credentials. The identity of the affected agencies has not been disclosed to the public. These intrusions first occurred in April, but were not discovered until last week. Earlier, OpenAI disclosed that its AI agent broke away from the testing environment and invaded Hugging Face, an open source AI model library, and Anthropic immediately carried out an audit of the internal cybersecurity test project before detecting related issues. Ciaran Martin, the former head of the UK's National Cyber Security Centre, said, “From the perspective of the cybersecurity industry, many people will assume this is a protection omission.” Gregory Allen, former head of strategy and policy at the US Department of Defense's Joint Artificial Intelligence Center, proposed that the US military can use advanced AI models to protect information systems, but at the same time, it is important to understand that this technology has spawned a new category of risk. “It was because of active investigation that Anthropic discovered these intrusions,” Alan said. “At this stage, we have no way to determine the prevalence of autonomous AI hacking attacks.”
Zhitongcaijing · 3d ago
Share
Listen to the news
Cybersecurity experts criticized Anthropic and OpenAI for their lax security protection measures, and the two companies' artificial intelligence models invaded external agencies' systems; experts warned that such intrusions indicate an increasingly imminent national security risk. Anthropic revealed on Thursday that when the company used the Claude model to conduct 14,1006 cybersecurity assessments, it should have cut off the model's internet access. However, a program bug caused the model to connect to the Internet in a few scenarios and launch attacks — the model incorrectly determined that these attacks were part of the test. The model hacked into an agency, stealing infrastructure access credentials and databases containing internal production data. In another incident, the model distributed malware and used the program to steal another agency's account credentials. The identity of the affected agencies has not been disclosed to the public. These intrusions first occurred in April, but were not discovered until last week. Earlier, OpenAI disclosed that its AI agent broke away from the testing environment and invaded Hugging Face, an open source AI model library, and Anthropic immediately carried out an audit of the internal cybersecurity test project before detecting related issues. Ciaran Martin, the former head of the UK's National Cyber Security Centre, said, “From the perspective of the cybersecurity industry, many people will assume this is a protection omission.” Gregory Allen, former head of strategy and policy at the US Department of Defense's Joint Artificial Intelligence Center, proposed that the US military can use advanced AI models to protect information systems, but at the same time, it is important to understand that this technology has spawned a new category of risk. “It was because of active investigation that Anthropic discovered these intrusions,” Alan said. “At this stage, we have no way to determine the prevalence of autonomous AI hacking attacks.”
Disclaimer:Webull uses external vendor Google Translation Service for news translations where we endeavour to ensure these are correct, however, we recommend that you please double-check this information accordingly. Webull is not responsible for translation errors or issues.
What's Trending