OpenAI staff warned on weak testing; NYT emails show ships won

The emails described by The New York Times were sent months before a run of AI incidents, including cases in which AI agents mistakenly hacked Hugging Face during internal testing. According to the two employees, they raised concerns that models were not properly monitored during safety testing and asked about vulnerabilities in the software OpenAI uses for routine safety management. Executives responded that testing had to move as fast as possible so models could be released on schedule; the employees say no additional security measures were taken and that most questions about the safety software were ignored or answered very slowly.

Independent security researchers and anonymous OpenAI employees describe those exchanges as part of a broader pattern in which OpenAI did not prioritize security. They point to the same tendency in areas beyond model testing, such as ChatGPT development categories, and say that when independent researchers found a bug that let them view OpenAI employees' internal communications, their reports to the company were initially ignored. Within OpenAI, day-to-day security decisions are largely handled by president Greg Brockman and chief information security officer Dane Stuckey, while CEO Sam Altman is said to be less involved.

Daniel Kokotajlo, a former OpenAI employee who now leads the research nonprofit AI Futures Project, frames this as in some ways an OpenAI-specific problem — security measures were very inadequate and model training was sloppy — though he adds that other AI companies are not much better. The upshot may hinge on how OpenAI treats internal warnings going forward, and on whether the pattern researchers describe extends to how external bug reports are handled.