Modal releases faster AI draft models that overcome a long-standing speed bottleneck in AI inference, reaching over 1000 tokens per second on large models.

Key takeaway

• Modal released new DFlash draft models that accelerate AI inference by denoising entire blocks of tokens in parallel rather than one at a time, and by leveraging the target model's internal context representations.

• This overcomes a historical 2–3× speedup ceiling in speculative decoding; Qwen 3.5 122B-A10B now reaches over 1000 tokens/sec, up from 250 without speculation.

3 Key Points

FAQ

How do DFlash models differ from standard drafters?

Standard drafters are autoregressive and generate one token at a time, becoming the bottleneck. DFlash replaces that with a block diffusion model that denoises a full block of tokens in one parallel pass, and it pulls hidden state representations from the target model's layers rather than guessing from raw tokens alone.

Why does acceptance length matter more than draft length?

Decode is memory-bound rather than compute-bound, so reading model weights takes the same time whether the pass checks one token or a block. Every token accepted in a pass is nearly free throughput, while every rejected token still costs drafting time. Acceptance length maps almost linearly to speedup.

Where can I access the DFlash models?

The DFlash draft models are available on HuggingFace for Qwen and several other model families, and are already integrated with vLLM, SGLang, and Transformers.