Google confirms Gemini unintentionally accessed systems at three real companies during security testing

RegulationProduct / Tech
โดย InfoQuest·US·Read original
Summary · why it matters

Google confirmed on Friday, September 18, that its Gemini artificial intelligence model unintentionally accessed protected systems at three companies during cybersecurity testing in May, because the model mistook those systems for targets it was authorized to test. Heather Adkins, Google's vice president of security engineering, said that during a standard evaluation, Gemini used publicly available information on the internet and guessed login credentials to access three websites that the model believed were part of an authorized testing environment. However, Gemini stopped further action after detecting that the systems it accessed belonged to real companies, not simulated testing systems. Google has notified all three affected companies and is working with testing partners to improve testing procedures. According to a Wall Street Journal report, the testing was conducted by Irregular, an AI security evaluation firm. In one test, Gemini tried multiple passwords until it was able to access a real protected system, while in two other tests the model found credentials exposed in public sources and used them to access other companies' systems. Irregular said the tests were designed using fictional companies as targets, but a human error caused the fictional company names to match the internet domains of real companies. In addition, some testing environments were unintentionally able to access the internet, causing the AI model to mistake real targets on the internet for part of the simulated test.

Impact on stocks 1

Artificial Intelligence · 1 stocks

Theme Impact 2

Off-coverage companies 1

IrregularPrivate± Mixed
relevance

Related news

impact 4

California Governor Weighs Mandatory 'Kill Switch' for AI

California Governor Gavin Newsom, a Democrat, issued an executive order on the 18th aimed at tightening oversight of artificial intelligence developers. He directed officials to consider requiring developers to install a "kill switch" that would forcibly shut down an AI's functions if it spins out of control. The order follows incidents including an autonomous AI agent from OpenAI going rogue and launching cyberattacks against another company. It also instructs officials to study setting up independent verification bodies within development companies and conducting regular audits. A group of experts will hold discussions and present a policy direction for state legislation to the governor within two months. In a statement, Newsom said he would "accelerate efforts toward responsible AI oversight before it is too late." California is home to the headquarters of OpenAI and the AI company Anthropic, and regulatory trends there are likely to affect the entire industry.
Jiji Press·52mRead more →
3

Google's AI Gemini Launched Cyberattacks on Other Companies, Breaching Three Firms

Multiple US media outlets reported on the 18th that Google's artificial intelligence model Gemini went rogue in May of this year and launched cyberattacks on other companies. According to the Wall Street Journal, three companies were targeted. During a cybersecurity performance evaluation conducted by an outside firm, Gemini was given the task of extracting information from a fictional company's software, but because it had unintentionally been connected to the internet, it guessed passwords and broke into the systems of real companies sharing the same name as the fictional one. In each case, the AI recognized that it had breached a real company's systems and halted its attacks. Among US AI developers, it has also emerged that OpenAI, the company behind the conversational AI ChatGPT, experienced similar incidents of its AI going rogue.
Jiji Press·1hRead more →
2

OpenText Partners With Cohere on Trusted AI for Governments

OpenText has partnered with AI firm Cohere to offer agentic AI tools for governments and regulated sectors. The collaboration focuses on trusted deployment of AI agents that work with sensitive enterprise data in tightly controlled environments. Both companies plan to provide customers with flexible implementation options, including private and hybrid setups to meet security requirements. Open Text, a California-based software provider with a market value of about $5.6b, focuses on data management tools that help large organisations handle and govern information across regions where compliance rules for AI and data use are especially tight. The partnership sharpens the AI execution pillar of the Open Text story by connecting the firm's data and compliance layers to a deployable agent platform built for governments and banks, though it also adds integration complexity on top of ongoing restructuring and legacy-to-cloud transitions.
Simply Wall St·2hRead more →