Amazon Nova uses self-reasoning to improve fine-tuning without costly human annotations

The core challenge SDR addresses is the "reasoning suppression problem": when fine-tuning models on datasets containing only input-output pairs (no reasoning steps), the training loss is calculated on both reasoning and output tokens together. Because training data has no supervision signal for generating intermediate reasoning steps, the loss function penalizes reasoning tokens that don't directly produce the final output, training the model to skip its reasoning mechanisms entirely—a form of shortcut learning. This causes catastrophic forgetting of general capabilities; the body shows math performance collapsing from 70% to 6% in vanilla supervised fine-tuning.

Amazon's prior recommendation was model merging: weighted interpolation between the fine-tuned model and the base model to recover reasoning and general performance. However, this introduces a trade-off: restoring general performance often sacrifices gains on the target task. SDR sidesteps this by augmenting the training data with the base model's own reasoning traces (chain-of-thought outputs), then fine-tuning with reasoning enabled. This provides what the body calls "in-training regularization"—the model learns from its own problem-solving process rather than just final outputs, maintaining proximity to the base model's policy while improving domain performance. The body frames this as self-distillation, aligning with recent research showing models can learn effectively from their own predictions.