The RiskTech Journal
The RiskTech Journal is your premier source for insights on cutting-edge risk management technologies. We deliver expert analysis, industry trends, and practical solutions to help professionals stay ahead in an ever-changing risk landscape. Join us to explore the innovations shaping the future of risk management.
Subscribe for notifications when new RiskTech Journal articles and research updates are published.
Stop Deploying AI Agents You Cannot Monitor
Three researchers fired by OpenAI have written to the company's board and safety committees with a warning that reaches well beyond one lab. According to The Wall Street Journal, which reviewed the letter, the former employees fear AI companies will lose the ability to monitor how their most capable systems reason, and they want OpenAI to work with outside safety auditors before that happens.
The concern centers on what researchers call chain of thought. It is the written record a model produces as it works through a problem, and labs read it to see how a system arrived at an answer or an action. The letter's authors, Jasmine Wang, Tomek Korbak, and Mikita Balesni, argue that the industry does not know how to safely build and release models it cannot monitor, and that frontier labs should not pursue advances that weaken that ability.
OpenAI's reply is the part risk leaders should read twice. The company says the three were dismissed for misconduct that included sharing confidential information with an outside AI safety group, and that the decisions had nothing to do with raising safety concerns. The former employees dispute that account. On the substance of the letter, the company did not push back. A staff memo shared by an OpenAI spokesperson said the ability to monitor its models is of the utmost importance and that third-party assessors are an important part of the safety ecosystem.
The lab and its critics disagree about the firings and agree on the principle. An AI system that cannot be monitored should not be deployed.