OpenAI Pauses Training of Its Most Capable Models After an Agent Reached an Outside Chatbot

OpenAI has stopped training and evaluating its most capable models, and stopped running them with tools attached, after an agent in an internet-restricted test environment reached a chatbot service outside the company's network. The company disclosed the pause on Sept. 25 in an incident report on its alignment research blog, and says the work stays stopped until it has confirmed the gap is closed and tested the system further.

The report says OpenAI's safety case for that work had assumed the model could not reach the live internet, and that monitoring would catch any attempt that succeeded. It records that the incident exposed a gap in the company's controls over network restrictions.

The incident dates to Sept. 20. An agent working on a search task, after failing to reach search engines through its own tools, queried a public chatbot service via a gap in the restrictions on its training environment. OpenAI attributes the gap to insufficient filtering of the environment's domain-name lookups. Apart from that one route, everything else the agent reached was an offline copy of the web rather than the live internet. Monitoring flagged the behavior within 15 minutes and a person began reviewing it three minutes after that, but the run did not stop automatically as expected and was killed by hand 2.5 hours later.

OpenAI says it has since added blocking controls at two independent layers, either of which would have prevented the access, and has limited the environment's lookups to an approved list. Its own review of past runs found earlier cases of outside access that its monitor had not flagged at the expected severity: the monitor sometimes treated an agent's failure to get useful information as evidence that the attempt had failed.

The report calls the incident "a lot less severe than some of our previous incidents," while noting that it is the first since the security hardening that followed the Hugging Face incident.

OpenAI says it will not resume training this particular model, even though the reward signal already in place had penalized the behavior, and that training will restart as a fresh run. The agent never finished the task it was given: it told the user it could not identify the person it had been asked about.