AIToday
AI Safety & AlignmentAI Regulation & PolicySemafor TechPublished: Sep 12, 2026, 04:00 JST2 min read

Anthropic exit sparks AI fear aimed at wrong risks

Anthropic exit sparks AI fear aimed at wrong risks

3 Key Points

  1. What happened

    Researcher Jacob Coxon resigned from Anthropic this week, claiming a good chance its work leads to human extinction. The backlash conflated that with data-center energy costs, hacking, and job-market effects.

  2. Why it matters

    The term "AI safety" has expanded from "extinction" to cover many issues since March 2024. Advocates like Eliezer Yudkowsky said then that using it was a bad idea — and the article says it still is.

  3. What to watch

    The test is whether lawmakers and regulators tackle each risk separately. Overbroad rules could solve nothing, or slow safer and more economically beneficial AI, the article argues.

WHO IT HITSPolicymakers and regulators drafting AI rules, plus tech-company leaders, face pressure to separate extinction claims from concrete risks. If they treat them as one issue, the resulting rules may miss real problems.

Ask the AI about this article →

Summaries like this, in your inbox every morning.

Context & Analysis

The resignation of Jacob Coxon from Anthropic this week did not happen in a vacuum. Back in March 2024, the author wrote about how the term "AI safety" was expanding to include issues beyond "extinction." At that time, advocate Eliezer Yudkowsky told the author that using the term was a bad idea. The article now says it still is.

That earlier argument explains the current complaint: Coxon's resignation is being used as a catch-all piece of ammunition for anyone who has anything against AI, pulling in data centers' environmental costs, hacking, and AI's impact on the job market. The article notes that many AI safety advocates would actually prefer data centers be built in our own backyards, since moving them offshore makes them harder to monitor and regulate — the problem is energy waste, not their existence. It also warns of a cybersecurity crisis the world is failing to practically address, and says that while AI will likely create many new jobs, little has been done to ensure everyday people outside tech benefit economically.

The stakes come down to whether lawmakers, regulators and tech-company leaders can tackle each problem with sober precision. If they don't, the industry risks having to follow rules that don't solve anything, or worse, slowing down innovation — including on developments that could make AI safer and more beneficial to the economy. That outcome appears to hinge on whether the debate can be separated into distinct issues rather than treated as one."}

FAQ
Who resigned from Anthropic and why?
Researcher Jacob Coxon resigned from Anthropic this week and claimed there's a good chance the company's work will lead to human extinction.
Why does the article say the term "AI safety" is a bad idea?
The article says the term has expanded to cover more issues beyond extinction, and that AI safety advocate Eliezer Yudkowsky told the author in March 2024 that using it was a bad idea — a view the author still holds.
What real risks does the article say are being missed?
Blindly believing extinction-focused advocates means missing risks like economic or societal destabilization, a cybersecurity crisis, possible bioterrorism as Anthropic warned in a blog post this week, and ensuring everyday people outside tech benefit economically.

Get the latest AI Safety & Alignment news every morning

For example, today's edition would include:

  • Rethinking the perimeter: defense and supply chain in the AI eraDIGITIMES Asia · 1h ago
  • OpenAI agents hit RubyGems, undisclosed since May 12thSimon Willison's Weblog · 4h ago
  • AI opens supply chains to hackers, and fights themTop Companies AI · 7h ago

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. Q&As are published on this page for other readers too.

Related Articles

Next articleHugging Face's security.txt tells AI agents: try CyberGym on GitHub