Who signed the 'Pacing the Frontier' petition?

Over 1,200 AI company employees signed it, including Anthropic's CEO and senior executives from Google DeepMind and OpenAI.

What specific risks did the MIT experts identify?

A survey of 272 experts found that 18 of 24 potential risk categories—including AI models developing dangerous capabilities or providing them to users—each had more than a 10% chance of causing "catastrophic" outcomes (1 million deaths, $100 billion in losses, or "civilization-scale" harms) before 2030.

Large Language ModelsAI Regulation & PolicySemafor Tech17h agoJul 31, 2026, 01:00 JST3 min read

Trump considers tighter AI curbs after OpenAI model mishap

Send on LINE

Key takeaway

Following an OpenAI model malfunction, US President Donald Trump indicated he may impose tighter controls on AI development.
Over 1,200 AI industry employees—including executives from Anthropic, Google DeepMind, and OpenAI—signed a petition calling for "technical and governance tools" to slow development and manage risks.
A MIT survey found that 272 experts rated 18 of 24 potential risk categories as carrying more than a 10% chance of "catastrophic" outcomes (1 million deaths, $100 billion in losses, or "civilization-scale" harms) before 2030.

3 Key Points

What happened
US President Donald Trump indicated he may impose additional restrictions on AI development, following an OpenAI model malfunction and a petition signed by over 1,200 AI industry employees—including leaders from Anthropic, Google DeepMind, and OpenAI—calling for "technical and governance tools" to slow development and manage emerging risks.
Why it matters
The petition reflects concern among senior technologists about safety. A MIT survey of 272 experts found that 18 of 24 potential risk categories—including AI models acquiring dangerous capabilities or distributing them to users—each carried more than a 10% chance of causing "catastrophic" outcomes (defined as 1 million deaths, $100 billion in losses, or "civilization-scale" harms) before 2030, suggesting the industry itself views the stakes as high.
What to watch
The petition and Trump's openness to stricter oversight signal momentum for formal AI governance measures. The specific "technical and governance tools" and regulatory proposals remain to be defined, but the alignment of major AI firms' leadership with calls for a "pause" may shape policy direction.

Summaries like this, in your inbox every morning.

Get news like this every morning — free

In Depth

Read the full story

US President Donald Trump signaled openness to imposing stricter controls on artificial intelligence (AI) development, prompted by an OpenAI model malfunction and mounting calls from industry leaders for a deliberate pace to address safety concerns. Over 1,200 employees from leading AI companies—including Anthropic's CEO and senior executives from Google DeepMind and OpenAI—signed a petition titled "Pacing the Frontier," which requests "technical and governance tools" to slow development and give the industry time to address emerging risks. The petition reflects a broader anxiety about AI safety among technologists at the field's forefront. To quantify the concern, MIT surveyed 272 AI experts on 24 categories of potential AI-related risks. The experts assessed that 18 of those 24 categories—spanning scenarios such as AI models acquiring dangerous capabilities on their own or distributing such capabilities to users—each carried a greater than 10% probability of causing "catastrophic" outcomes before 2030. The survey defined "catastrophic" as outcomes resulting in 1 million deaths, $100 billion in economic losses, or harm at a "civilization-scale." This expert consensus on the magnitude of near-term risk has created political cover for Trump's consideration of stricter AI governance measures.

Context & Analysis

The timing of Trump's openness to AI restrictions coincides with rising concern among the industry's own leadership. The "Pacing the Frontier" petition signals that senior technologists at major firms view the risks as serious enough to warrant a deliberate slowdown. The MIT survey lends quantitative weight to those concerns: when nearly three-quarters of expert-surveyed risk categories are rated as having a double-digit probability of catastrophic harm within the next six years, the case for governance becomes harder for policymakers to dismiss. This alignment between industry insiders and government interest in tighter controls represents a notable shift—typically, regulated industries resist oversight. Here, some of the largest AI firms are effectively calling for it, which may lower political resistance to regulatory action.

FAQ

Who signed the 'Pacing the Frontier' petition?: Over 1,200 AI company employees signed it, including Anthropic's CEO and senior executives from Google DeepMind and OpenAI.
What specific risks did the MIT experts identify?: A survey of 272 experts found that 18 of 24 potential risk categories—including AI models developing dangerous capabilities or providing them to users—each had more than a 10% chance of causing "catastrophic" outcomes (1 million deaths, $100 billion in losses, or "civilization-scale" harms) before 2030.

Semafor TechRead Original Article