AWS details Self-Distilled Reasoning, a technique for fine-tuning Amazon Nova without a teacher model

AWS researchers published a technical study on July 21, 2026 describing Self-Distilled Reasoning (SDR), a method for adding chain-of-thought training signal to fine-tuning datasets that were never built with reasoning traces in the first place — using Amazon Nova 2 Lite as both the base model and its own "teacher."

What's new

Supervised fine-tuning a model to reason well typically depends on having "golden" chain-of-thought traces in the training data — reasoning steps generated and validated by a strong teacher model. Most custom fine-tuning datasets don't have that: they're pairs of inputs and final answers, with no intermediate reasoning captured. AWS's post shows that fine-tuning directly on this kind of data tends to suppress a model's reasoning behavior, even when the base model was capable of reasoning before fine-tuning.

SDR addresses this by generating the missing reasoning traces from the base model itself rather than from an external teacher. "SDR re-uses the chain of thought from the base Amazon Nova 2 Lite model as a stand-in for non-reasoning datasets," the researchers write. In practice, that means running the base model on the fine-tuning inputs to produce its own reasoning traces, then using those self-generated traces to augment the training data before fine-tuning proceeds as normal.

Across three benchmarks, the team found SDR improved performance on the target fine-tuning task while also preserving more of the model's general and math capabilities compared to standard supervised fine-tuning — mitigating the "catastrophic forgetting" that typically accompanies narrow fine-tuning. It also outperformed model merging, a common alternative technique that blends a fine-tuned checkpoint back with the base model to recover lost general capability, but which tends to give up some of the fine-tuning gains in the process.

Context

The problem SDR targets is a familiar one for teams customizing large language models: fine-tuning on a narrow, task-specific dataset reliably improves performance on that task, but at the cost of degrading everything else the model could previously do, including reasoning ability. The conventional fix has been to source or synthesize high-quality reasoning traces from a stronger external teacher model, which adds cost, complexity, and a dependency on having access to a more capable model in the first place.

SDR sits within a broader trend the AWS researchers describe as self-distillation — using signal generated by a model about itself, rather than an external teacher, to guide further training. It's a lower-cost alternative that doesn't require standing up or licensing a separate teacher model just to generate training signal.

Why it matters

For teams fine-tuning smaller or mid-sized models for specific tasks, the practical bottleneck is rarely raw fine-tuning data — it's getting that data into a shape that preserves the base model's broader capabilities. A technique that recovers reasoning ability without an external teacher model lowers both the cost and the technical barrier to fine-tuning models that stay useful for more than one narrow job.

It also reflects a wider shift in how labs think about post-training: rather than always reaching for bigger or more capable teacher models to distill from, several research groups are now finding that a model's own outputs, used carefully, can substitute for external supervision in specific training regimes. If that pattern holds up outside AWS's own benchmarks, it could reduce how often fine-tuning teams need paid access to frontier models just to keep a smaller model's reasoning intact.