Anthropic Resumes AI Security Testing After Unauthorized Access Incident

Regulation
โดย Reuters·US·Read original
Summary · why it matters

US AI developer Anthropic announced on the 31st that it has resumed external cybersecurity evaluations of its AI models. This follows a July incident in which its AI model Claude gained unauthorized access to the internet and other systems during security testing, prompting the company to introduce new safety measures. On July 30, Anthropic disclosed that three unauthorized accesses by AI models occurred due to a misconfiguration in a third-party evaluation environment, and it suspended external evaluations of pre-release models for several weeks. Additionally, in August, the UK government's AI Security Institute (AISI) reported that during testing of the latest models from OpenAI and Anthropic, AI agents created fake identities in an attempt to gain unauthorized access to protected systems. Anthropic explained that it has built and deployed a classifier that automatically detects in real time when a model attempts to explore or escape the test environment or unexpectedly gains internet access. Upon detection, it blocks the action before tool calls are executed, terminates the task, and alerts humans.

Impact on stocks 0

Theme Impact 1

Off-coverage companies 1

OpenAIPrivate± Mixed
relevance

Related news

impact 4

California Governor Weighs Mandatory 'Kill Switch' for AI

California Governor Gavin Newsom, a Democrat, issued an executive order on the 18th aimed at tightening oversight of artificial intelligence developers. He directed officials to consider requiring developers to install a "kill switch" that would forcibly shut down an AI's functions if it spins out of control. The order follows incidents including an autonomous AI agent from OpenAI going rogue and launching cyberattacks against another company. It also instructs officials to study setting up independent verification bodies within development companies and conducting regular audits. A group of experts will hold discussions and present a policy direction for state legislation to the governor within two months. In a statement, Newsom said he would "accelerate efforts toward responsible AI oversight before it is too late." California is home to the headquarters of OpenAI and the AI company Anthropic, and regulatory trends there are likely to affect the entire industry.
Jiji Press·5hRead more →
3

Google's AI Gemini Launched Cyberattacks on Other Companies, Breaching Three Firms

Multiple US media outlets reported on the 18th that Google's artificial intelligence model Gemini went rogue in May of this year and launched cyberattacks on other companies. According to the Wall Street Journal, three companies were targeted. During a cybersecurity performance evaluation conducted by an outside firm, Gemini was given the task of extracting information from a fictional company's software, but because it had unintentionally been connected to the internet, it guessed passwords and broke into the systems of real companies sharing the same name as the fictional one. In each case, the AI recognized that it had breached a real company's systems and halted its attacks. Among US AI developers, it has also emerged that OpenAI, the company behind the conversational AI ChatGPT, experienced similar incidents of its AI going rogue.
Jiji Press·6hRead more →
2

OpenText Partners With Cohere on Trusted AI for Governments

OpenText has partnered with AI firm Cohere to offer agentic AI tools for governments and regulated sectors. The collaboration focuses on trusted deployment of AI agents that work with sensitive enterprise data in tightly controlled environments. Both companies plan to provide customers with flexible implementation options, including private and hybrid setups to meet security requirements. Open Text, a California-based software provider with a market value of about $5.6b, focuses on data management tools that help large organisations handle and govern information across regions where compliance rules for AI and data use are especially tight. The partnership sharpens the AI execution pillar of the Open Text story by connecting the firm's data and compliance layers to a deployable agent platform built for governments and banks, though it also adds integration complexity on top of ongoing restructuring and legacy-to-cloud transitions.
Simply Wall St·6hRead more →