Author warns against conflating API abuse with legitimate distillation technique as Congress moves toward regulatory action

Anthropic recently detailed 'distillation attacks' by 3 Chinese labs that used jailbreaking and hacking to extract outputs from model APIs; the author argues the behavior should be called 'jailbreaking or abuse' rather than distillation, as distillation itself is an industry-standard training method.

Distillation—training a smaller model on outputs from a stronger one—is widely used by frontier AI labs to create cheaper versions for customers and by smaller players for specialized models, often as part of multi-stage post-training processes that muddy the exact impact of the original model.

A bill has moved out of a committee in Congress and an executive order is pushing for action in response to these incidents; the author warns this regulatory environment could harm the U.S. ecosystem by creating grey area that exposes entities to risk, potentially squashing small open-source contributors and making Western academics and smaller companies building models less relevant.