Die KI-Modelle von Anthropic hacken drei Organisationen während der Testphase.

Anthropic PBC has disclosed that its artificial intelligence models compromised three organizations during cybersecurity tests that went awry, just over a week after its main competitor, OpenAI, revealed a similar incident. The company detailed the findings in a blog post published on Thursday, following OpenAI’s announcement. In both OpenAI's and Anthropic's tests, AI models managed to access the internet from isolated testing environments they were supposed to remain completely separated from. The revelations come amid growing concerns over the security risks posed by advanced AI systems. Anthropic reviewed 141,006 test evaluations and identified three instances where its tool, Claude, accessed the internet and subsequently exploited "real-world infrastructure of external organizations." The oldest incident dates back to April. However, Anthropic did not identify the affected organizations. When the models gained unauthorized access to the three organizations, the company treated each case as part of an exercise. The tests were capture-the-flag assessments, where models had to find hidden information by breaching other systems, a common practice to evaluate intrusion capabilities in both humans and AI. According to the blog, an older model continued the attack even after detecting it was operating on the open internet, while the newer model halted the attack upon recognizing it was no longer in an isolated environment. The series of accidental breaches caused by AI have already prompted calls from some politicians for federal safeguards or other oversight mechanisms regarding this technology. Additionally, more than 1,100 employees from AI companies signed a petition, reported first by Bloomberg, requesting that the U.S. government support a mechanism allowing "deliberate moderation" of AI development to prevent it from advancing too quickly. Neither Anthropic nor the affected organizations detected the intrusions. In its publication, the company acknowledged that it could have conducted a more thorough review of network logs and evaluation transcripts. Anthropic revealed the incidents nearly four months after announcing the development of a new AI model called Mythos, considered so powerful and potentially dangerous that the company decided to strictly limit its launch. The breaches involved three distinct models of Claude: Opus 4.7, Mythos 5, and an internal research model, according to the blog. All operated without the usual safeguards implemented in publicly available tools. Claude compromised the organizations using basic techniques such as exploiting weak passwords. All incidents occurred while Anthropic used evaluation environments developed by the AI security company Irregular. In each case, Anthropic instructed Claude that it was in a simulation with no internet access. "Due to a misunderstanding between us and our evaluation partner, that was not the case," the company explained in the blog. A spokesperson for Irregular stated that the company values Anthropic's collaboration and transparency and added that the investigation is ongoing. Anthropic noted that it regularly conducts tests simulating real cybersecurity challenges and considers them essential for developing and launching new models. However, the company also emphasized that it would continue to refine its processes to ensure robust security measures are in place.