Apple researchers propose learned policies for diffusion language model sampling

What happened

Apple researchers published work on training sampling procedures for diffusion language models (dLLMs) using reinforcement learning. Instead of relying on manual heuristics like confidence thresholding, they developed a lightweight policy based on a single-layer transformer that decides which tokens to unmask at each step.

Why it matters

dLLMs promise efficiency gains during inference by decoding multiple tokens in parallel, but their sampling strategy—which tokens to reveal—has relied on hand-tuned heuristics that require manual adjustment and degrade with larger block sizes. The trained policies match state-of-the-art heuristics in block-wise generation and outperform them in full-diffusion settings, offering a more automated and potentially more scalable approach.

What to watch

The work demonstrates that recycling computation from discarded tokens is beneficial, and the researchers note that dLLMs' global planning and iterative refinement features are particularly useful for code generation—a domain where decoding behavior remains under-explored.