New DIVERSED technique boosts language model inference speed by relaxing token verification constraints through dynamic ensemble blending.

Speculative decoding accelerates LLM inference by drafting multiple tokens in parallel, but verification bottlenecks limit speedup gains

Current methods reject many plausible tokens because they strictly enforce exact distribution matching with target models, reducing acceptance rates

DIVERSED framework uses ensemble-based verifier that blends draft and target model distributions with task-dependent and context-dependent weights

Approach includes theoretical justification and demonstrates substantially higher inference efficiency compared to standard speculative decoding methods

Relaxed verification preserves generation quality while improving time efficiency for large language model deployment