Anthropic researcher quits, warns of 'gambling with our lives'

The resignations and warnings from within Anthropic and OpenAI highlight a deep split in the AI industry over how to handle the pursuit of superintelligence. Jacob Coxon's departure is notable because he moved from OpenAI to Anthropic, a lab that reportedly sees itself as the responsible actor in this race. His critique that Anthropic is still racing to be first, despite its self-image, suggests that even labs trying to be careful may be caught up in competitive pressures.

The statement from top executives at Anthropic, DeepMind, and OpenAI calling for government intervention shows that these concerns reach the highest levels. However, the call for 'government-backed tools' is vague and has not been translated into concrete policy. The gap between public statements and actual behavior, as Coxon suggests, may be the crux of the problem.

The stakes, as articulated by the unnamed Anthropic researcher, are existential, with a personal probability estimate of over 10% that AI could cause human extinction within a decade. Whether these warnings lead to meaningful changes in development pace or remain as internal dissent could determine how the industry navigates its most significant challenge.