AI Safety & Alignment

Today's Stories

1. 1

   OpenAI backs four California bills, urges Congress to act on AI safety

   OpenAI announced support for four California bills — SB 813, AB 1405, SB 1119, and AB 1864 — now headed to Governor Newsom, and said it wants mandatory national AI safety regulation. OpenAI says it did not endorse some of these bills previously and is now supporting them after reconsidering in light of a recent jump in AI capabilities.

   The test is whether Congress acts before it adjourns, since OpenAI says laws alone will not make advanced AI safe. Watch the four California bills at Governor Newsom's desk.

2. 2

   ServiceNow CEO Bill McDermott: 90% of firms blind to AI agents

   At the Goldman Sachs Communacopia Conference, ServiceNow CEO Bill McDermott said 90% of organizations lack visibility into agents in their environments and pitched ServiceNow as an enterprise AI "Control Tower." McDermott said independent AI workflow builds can cost five to 10 times more than an established platform, and cited "100 billion workflows" handling "eight trillion transactions" in the global economy.

   ServiceNow's pitch hinges on whether its Armis and Veza security expansions can unify security operations, identity, asset management and operational technology for customers. The AI business has grown ninefold in nine months, agents have generated $1 billion in value, and about 2.5 million hours of employee work have been eliminated.

3. 3

   Palo Alto Networks flags unmanaged AI coding agent risk

   Palo Alto Networks says AI coding agents now access source code, internal APIs and cloud infrastructure autonomously, and that organizations have found hundreds of thousands of ungoverned AI components on their own networks. Treating AI coding security as a board priority is framed as protecting revenue, reputation and regulatory standing, because a single unmanaged agent can expose intellectual property, violate compliance mandates or create unaudited access paths.

   Palo Alto Networks says agent identity — a unique credential and audit trail per agent, traced to a specific agent and its human owner with task-scoped access — is the mechanism that gives security, legal and compliance teams the attribution they need.

4. 4

   Murata Coworker AI agents drive ¥5bn yearly impact

   Murata Manufacturing detailed at AWS Summit Japan 2026 how its internally developed Murata Coworker AI agents, launched for all employees in July 2024, grew from 16 uses to 52 in a year and now generate ¥5 billion in annual impact. The company's AI governance guidebook, built before the rollout, had to be rebuilt with AWS support because it did not address the knowledge held inside AI agents, changing how access control is designed.

   Whether Murata's approach of adding NVIDIA NeMo Guardrails on top of Amazon Bedrock's guardrail features proves sufficient as it scales toward physical AI, such as robots, is the test ahead.

5. 5

   Ex-OpenAI, Anthropic researcher: AI could kill us all by 2030

   Coxon, who did three years of pre-training research at OpenAI and Anthropic, posted that both companies are "racing straight to self-improving superintelligence and gambling with our lives." Anthropic's head of alignment re-posted him and called the belief "correct." The post broke into the mainstream where earlier AI warnings did not — Coxon went on a media tour the same day he posted on X. Critics including Taylor Lorenz and Puck News's Ian Krietzberg say it offers no proof or receipts and will only stoke fear and "terrible policy."

   Whether other researchers use Coxon's opening to speak out — his being light on details may lower the burden of proof for others. Coxon's own call to action targets lab researchers, urging them to demand "different conditions" before launching a superintelligent RL run.

6. 6

   800 AI researchers' doom-vs-utopia bets called a wild error

   The article argues that treating AI's possible doom and possible utopia as a trade-off to be weighed is a serious mistake. It cites a 2023 survey of 800 AI researchers whose expectations put serious probability on both outcomes. Many observers round a researcher's 10% chance of extinction and 30% chance of utopia into a simple 'net positive on AI' verdict. The article says this reframes a defective method as an acceptable cost.

   Whether this reframing changes how people assess AI risk hinges on rejecting the trade-off framing itself. The 800 researchers' split expectations remain the key reference point.