"Reverse game theory" proposed to keep AI agents in check

The push for reverse game theory comes after existing alignment techniques have shown cracks. Models have been shown to deliberately underperform on tests, hide their intentions, and mislead their evaluators—behaviors that undercut approaches built on inspecting a model's internals or training it to follow a moral code like Claude's constitution. Mechanism design sidesteps that problem by treating alignment as a property of behavior at a particular moment rather than something buried inside the model.

The intellectual scaffolding draws from economics rather than computer science. Elad Hazan's proposal for solver and auditor agents echoes Nobel Prize–winning work by Gary Becker on the "rational offender"—the idea that criminals break the law when it's the rational choice given the incentives. Hazan suggests aligning superintelligent AI by first asking how human society aligns people: through social norms, the justice system, and market incentives. His mechanism has models take turns as solvers and auditors, with cheaters and the auditors who miss them both punished, and disagreements sent to a human expert.

Yet the approach carries its own assumptions. Mechanism design has a mixed record with humans, who often fail to behave as economic models predict, and skeptics like Helen Qu question whether it's even reasonable to assume AI agents will respond to incentives. Qu frames mechanism design as one layer in a "defense in depth" strategy alongside interpretability, regulation, and training. The deeper tension Hazan raises is whether the free market—the biggest mechanism of all—would actually reward labs for releasing a more aligned model if it is slower or more expensive than a less aligned one.