Jack Clark, Jakub Pachocki join AI 'intelligence explosion' warning

The paper arrives amid signs that safety discussions between labs are already underway. Bloomberg reported on Sept. 15 that OpenAI policy chief Chris Lehane said discussions with Anthropic and Google DeepMind had been running for several weeks, and the paper itself proposes controls such as capability limits and embedded auditors.

Anthropic's number helps show what is at stake. AI completed 26% of its internal AI research and development work in August 2026 under human supervision, up from 1% in March, according to GovAI's summary. The authors also estimate that under full automation, continued research gains, and no additional bottlenecks, progress could accelerate roughly tenfold within 1.5 years — meaning a year of today's progress would take about five weeks.

The value of the proposed oversight hinges on how it is implemented. For embedded evaluators, the key questions are when they begin work, what they can inspect, and how redaction rules affect their ability to publish troubling findings — details the body leaves open. Meanwhile, the U.S. and China agreed Sept. 26 to establish an AI incident communication channel and hold a dialogue in November, even as Trump rejected slowing U.S. AI efforts, leaving a gap between Washington's position and the calls for restraint from inside AI labs.