Anthropic's Kernion: Claude writes for AI models, not people

Anthropic employee Jackson Kernion, who works on Claude fine-tuning, recently explained why Opus 4.6 was the last good writing model from the company. His answer points to a trade-off inside the training process itself: models are improving fast at math, code, and reasoning, but their writing quality has stalled or even gotten worse.

The mechanism Kernion describes is about who the model is writing for. Newer Claude models have been optimized for math and code, but they have also been trained to produce technical explanations aimed at other AI models, which he calls being "adapted to LLM psychology." He compares this to humans who only communicate with other autistic people and develop a style that works within that group but feels hard to follow for outsiders. Because LLMs have far more working memory than humans and pick up on details at a much finer level, a writing style emerges in training that works well for AI models but reads to people as overly-dense info dumps.

The fix, according to Kernion, comes down to the reward structure in reinforcement learning. Some rewards optimize for AI model comprehension, others for human comprehension, and the more you train on math and code, the more you have to actively push back by rewarding simple explanations. With Opus 5.5, Anthropic found a better balance, and Kernion says he hasn't been as happy about a model's writing since Opus 4.6. He notes, though, that Opus 5.5 does not surpass the older model, and that it is a hard problem Anthropic will continue to work on. The stakes appear to hinge on whether the company can reward human-readable explanations without giving up the math and coding gains that come from writing for other models.