TS-DFM student hits 32% lower perplexity in 8 steps

Discrete flow matching builds text by iteratively turning noise tokens into coherent language, but it can need hundreds of forward passes. Autoregressive models sidestep that cost by generating one token per pass, yet they are inherently serial, which caps throughput and inflates latency on long sequences. Diffusion language models parallelize across positions, but standard discrete diffusion typically needs hundreds to thousands of model evaluations to reach high quality.

The Apple team argues the usual explanation for a weak student — insufficient capacity — is backwards. Each training trajectory, they write, is built through a chain of blind stochastic jumps with no evaluation of sequence quality, and one bad decision at an early midpoint propagates through later steps while the student must imitate the result. TS-DFM replaces those blind jumps with guided navigation.

The stakes hinge on whether the 8-step, 170M-parameter result generalizes. The gains were consistent across source distributions and three evaluators of increasing scale, and TS-DFM beat baselines trained on 6× more data or using 5× larger models — but the demonstration remains at 170M parameters, so its reach at larger scale is the open question.