Anthropic's new paper shows AI improving its own alignment

The paper, led by Anthropic fellow Chen Yueh-Han, describes a system that automates research tasks traditionally done by humans. It searches literature, proposes methods, and trains models in 30-minute cycles, keeping what works and discarding what doesn't. This mirrors the iterative nature of human-led research but at a scale and speed that humans cannot match.

The researchers are direct about the implications. The system beats what experienced humans propose on average within six hours, and the cost difference is stark: roughly $4 per hour in API inference compared with $150 per hour for human researchers. These numbers suggest that automating alignment research could be significantly more efficient than relying on human expertise alone.

The paper's limitations are also clear. The approach depends on benchmarks accurately capturing alignment goals, and the quality of the literature the system draws from. Establishing and maintaining those benchmarks remains a substantial challenge. Still, the paper argues these results provide early evidence that automated alignment post-training could become practical in the near term.