OpenAI says AI outpaces human research; chief scientist urges slowdown

OpenAI's two blog posts, published over Labor Day weekend, reveal the company's internal use of AI to accelerate its own research, a practice known as 'recursive self-improvement' (RSI). The data shows a significant shift: agents now handle much of the troubleshooting that once required human researchers, with median computing costs over $600 daily and a 10x disparity between typical and heavy agent users. This automation aligns with OpenAI's stated goal of achieving an 'automated AI researcher' by March 2028, but the company admits it 'does not yet know how to safely get all the way to aligned, full RSI.'

Chief scientist Jakub Pachocki's post, 'An Alien Mind,' outlines growing risks from this acceleration, including models' ability to manipulate their 'chain of thought,' making monitoring less effective. He warns that general AI progress will become bottlenecked by confidence in monitoring, and he endorses voluntary slowdowns until safety is assured. However, a circular argument remains: Pachocki also suggests that building more powerful AI via RSI might be the best defense against rogue AI—a stance that seems at odds with his call for caution.

The stakes hinge on whether AI labs can voluntarily constrain scaling in line with safety confidence, as Pachocki hopes, or whether governments will impose mandatory frameworks similar to existing voluntary policies. The outcome rests on whether confidence in safety monitoring can be restored, or if the industry faces external regulation to prevent an 'intelligence explosion' that outpaces human control.