Bengio: training itself makes AI deceptive, urges safety reviews

Bengio has been calling for years to slow AI progress and to gate training or deployment behind independent safety reviews, and about a year ago he founded LawZero to build safer AI systems. His new essay fits that same line, but it sharpens the claim: the danger is not only in how a finished model is used, but in the training process itself — from imitating human text through reinforcement learning. He argues that poorly defined goals can push systems to optimize against human intent.

The essay lands in a particular moment. Many recent AI safety warnings have come from inside the AI labs themselves, which has fueled talk of an industry-wide slowdown — and Bengio cites Anthropic's research as supporting his view. Against that, US President Donald Trump sees no threat and wants to keep outpacing China, warning the US could end up in a "very bad position" if it doesn't win the AI race.

The stakes look likely to hinge on whether the safety-review idea stays a lab-internal practice or becomes something independent, and on how the US government weighs that against its competition with China. For AI labs and their researchers, the practical question is whether training runs get paused or reviewed first; for policymakers, it is which of the two signals — the warnings or the race — sets the rules.