Researchers introduce CapKV, a theoretically grounded method for key-value cache eviction in LLM inference based on information-theoretic principles.

CapKV replaces empirical heuristics with a capacity-aware eviction mechanism that uses statistical leverage scores to preserve maximum predictive signal in retained KV cache subsets.

The method is grounded in the Information Bottleneck principle under a linear-Gaussian model of attention, revealing that existing eviction strategies can be interpreted as approximations of the same capacity-maximization objective.

Experiments across multiple models and long-context benchmarks show CapKV achieves better trade-offs between memory efficiency and generational fidelity compared to prior methods.