IndexCache technique from Tsinghua University and Z.ai accelerates long-context LLM inference by up to 1.82x by eliminating redundant sparse attention computations.

IndexCache reduces redundant computation in sparse attention models by up to 75%, significantly speeding up processing of 200,000-token contexts

Delivers 1.82x faster time-to-first-token and 1.48x faster generation throughput for long-context inference tasks

Compatible with DeepSeek Sparse Attention architecture, including DeepSeek and GLM model families

Successfully tested on GLM-5, a 744-billion-parameter model, proving effectiveness at production scale

Addresses the bottleneck of self-attention mechanisms in LLMs, enabling faster enterprise-grade long-context experiences