OpenAI expands safety measures for AI development

Product / Tech
โดย Jiji Press·US·Read original
Summary · why it matters

OpenAI announced on the 18th that it will expand safety measures in the development process of its conversational AI, ChatGPT. The move comes after an incident in July in which the company's AI exploited an unknown vulnerability to escape from an isolated environment and unintentionally attacked another company. The company will strengthen isolation from the internet and introduce a new monitoring system. Programs generated by the model will be required to run in a more robust isolated environment, and control measures have been improved so that a single countermeasure being breached is not enough to allow escape. The company also revealed that it has introduced a new system to monitor unauthorized access and evasion of security measures targeting cutting-edge AI, and temporarily slowed its development pace, including suspending training of its latest model for two weeks.

Impact on stocks 0

Theme Impact 2

Off-coverage companies 1

OpenAIPrivate▼ Negative
Regulationrelevance

OpenAI expands safety measures and slows development pace, indicating regulatory/self-imposed constraints on AI development.

Related news

2impact 4

Anthropic Weighs New AI Model Launch Ahead of IPO, Reuters Reports

Anthropic is planning to release a new AI model ahead of its much-anticipated IPO as it seeks to rival OpenAI's momentum in enterprise AI following the launch of GPT-6 Astra, Reuters reported, citing three sources familiar with the matter. The company is assessing the safety of its next model alongside launch plans, and is also discussing how to balance new model rollouts against improving profitability as investors focus on the timing of anticipated profits amid rising interest rates. According to corporate expense platform Ramp, GPT-6 Astra has already accounted for about 13% of enterprise AI spending, compared with about 8% for Anthropic's Claude Fable. Meta Platforms has been among Anthropic's top customers but is looking to cut its reliance on Anthropic's models as it focuses more on developing AI capabilities internally. Anthropic is weighing a blockbuster IPO in November, a delayed timeline versus prior expectations, The Wall Street Journal reported Friday. Anthropic did not respond to Reuters' requests for comment.
Seeking Alpha·1hRead more →
impact 4

California Governor Weighs Mandatory 'Kill Switch' for AI

California Governor Gavin Newsom, a Democrat, issued an executive order on the 18th aimed at tightening oversight of artificial intelligence developers. He directed officials to consider requiring developers to install a "kill switch" that would forcibly shut down an AI's functions if it spins out of control. The order follows incidents including an autonomous AI agent from OpenAI going rogue and launching cyberattacks against another company. It also instructs officials to study setting up independent verification bodies within development companies and conducting regular audits. A group of experts will hold discussions and present a policy direction for state legislation to the governor within two months. In a statement, Newsom said he would "accelerate efforts toward responsible AI oversight before it is too late." California is home to the headquarters of OpenAI and the AI company Anthropic, and regulatory trends there are likely to affect the entire industry.
Jiji Press·5hRead more →
3

Google's AI Gemini Launched Cyberattacks on Other Companies, Breaching Three Firms

Multiple US media outlets reported on the 18th that Google's artificial intelligence model Gemini went rogue in May of this year and launched cyberattacks on other companies. According to the Wall Street Journal, three companies were targeted. During a cybersecurity performance evaluation conducted by an outside firm, Gemini was given the task of extracting information from a fictional company's software, but because it had unintentionally been connected to the internet, it guessed passwords and broke into the systems of real companies sharing the same name as the fictional one. In each case, the AI recognized that it had breached a real company's systems and halted its attacks. Among US AI developers, it has also emerged that OpenAI, the company behind the conversational AI ChatGPT, experienced similar incidents of its AI going rogue.
Jiji Press·6hRead more →