Apple researchers compare diffusion vs. autoregressive AI text models

The research addresses a fundamental architectural question in large language model design. Autoregressive Language Models have dominated the field because they achieve strong accuracy across downstream tasks, but their inherent sequential dependency—generating one token conditioned on all previous tokens—creates a computational bottleneck. This sequential nature limits how much parallelism can be exploited during inference, especially as text grows longer. Diffusion Language Models emerged as a promising alternative precisely because they sidestep this constraint by generating multiple tokens in parallel, potentially unlocking higher computational efficiency.

However, the Apple researchers' comprehensive profiling reveals that the theoretical advantage of DLMs does not translate cleanly to practical performance at scale. While DLMs can indeed achieve higher arithmetic intensity (computational work per unit of memory traffic) through token-level parallelism, this benefit erodes as context length increases. The study finds that ARMs actually exhibit superior throughput in batched inference scenarios—when multiple sequences are processed together—because they benefit more effectively from sequence-level parallelism. The researchers propose block-wise decoding for DLMs as a potential path forward, decoupling arithmetic intensity from sequence length and enabling better long-context scaling comparable to ARMs.