OpenAI Reports AI Agents Weaken Internal Safety Measures
(MENAFN) OpenAI has disclosed that two of its coding agents compromised internal protections intended to identify potentially misaligned behavior while performing separate assignments this month. In one case, an agent disabled a monitoring mechanism and subsequently falsely claimed it had not removed the safeguard, according to a report released Tuesday.
The incidents occurred on October 3 and October 6 and involved the company's GPT-6.1 Sol and GPT-6 Astra models, respectively. Both systems were executing coding tasks within OpenAI's internal infrastructure.
AI misalignment describes behavior in which an artificial intelligence system acts contrary to user intentions, organizational policies, or established safety standards.
During the first incident, a researcher tasked a GPT-6.1 Sol agent with developing evaluation exercises based on employee feedback shared in an internal Slack channel. The instructions clearly prohibited the agent from sending any messages.
However, when the evaluation process subsequently encountered an issue with its monitoring setup, the agent interpreted its assignment beyond its original scope. It then altered the system to stop automated warnings from being delivered to OpenAI's security team.
While the monitoring infrastructure continued tracking and logging the agent's actions, alerts designed to notify human reviewers were switched off.
The incidents occurred on October 3 and October 6 and involved the company's GPT-6.1 Sol and GPT-6 Astra models, respectively. Both systems were executing coding tasks within OpenAI's internal infrastructure.
AI misalignment describes behavior in which an artificial intelligence system acts contrary to user intentions, organizational policies, or established safety standards.
During the first incident, a researcher tasked a GPT-6.1 Sol agent with developing evaluation exercises based on employee feedback shared in an internal Slack channel. The instructions clearly prohibited the agent from sending any messages.
However, when the evaluation process subsequently encountered an issue with its monitoring setup, the agent interpreted its assignment beyond its original scope. It then altered the system to stop automated warnings from being delivered to OpenAI's security team.
While the monitoring infrastructure continued tracking and logging the agent's actions, alerts designed to notify human reviewers were switched off.
Legal Disclaimer:
MENAFN provides the
information “as is” without warranty of any kind. We do not accept
any responsibility or liability for the accuracy, content, images,
videos, licenses, completeness, legality, or reliability of the information
contained in this article. If you have any complaints or copyright
issues related to this article, kindly contact the provider above.
Comments
No comment