Anthropic researcher Jacob Kochson quits, warns AI may be uncontrollable by next year

Jacob Kochson's resignation from Anthropic comes shortly after he moved from OpenAI, where he was a key contributor to GPT-4.5, to Anthropic, which he viewed as more safety-focused. His public criticism, aired on X and in the Wall Street Journal, reveals deep internal tensions within leading AI labs about the pace of development. He argues that both OpenAI and Anthropic are engaged in a race toward self-improving superintelligence, treating human lives as collateral, and that internal discussions already use terms like 'crunchtime' and 'endgame,' indicating a sense of urgency and perhaps fatalism.

Kochson's specific warning—that the most radical scenarios are becoming reality and that systems might be uncontrollable by the end of next year—underscores the existential stakes as he perceives them. He suggests that while researchers privately express fear, public statements remain measured. His call for researchers to question whether they should proceed with reinforcement learning without fully understanding system internals reflects a broader plea for responsibility.

Looking ahead, Kochson points to the OpenAI-Hugging Face incident as a potential catalyst for coordination among U.S. firms, but he doubts it will prevent a global race. He hints that binding measures, such as temporary bans on capability improvements, may be necessary. The outcome hinges on whether industry players and governments heed such warnings or continue on the current trajectory, a decision that will profoundly shape the future of AI development and its impact on humanity.