SSV framework combines speculative decoding and sparse attention to accelerate LLM inference on NVIDIA H100 GPUs

SSV (Sparse Speculative Verification) addresses a structural mismatch between speculative decoding (which reuses target-model computation across multiple queries) and dynamic sparse attention (which assigns query-specific sparse layouts), enabling better KV-block reuse and reducing branch-fusion overheads.

The framework uses overlap-aware grouped-query execution, refresh/reuse-based NSA kernel fusion, and profile-guided prompt-adaptive orchestration to improve cross-query reuse and select effective draft-verification strategies under user-specified precision classes.

Experiments on NVIDIA H100 GPUs show SSV achieves up to 3.49x end-to-end throughput over autoregressive NSA decoding and up to 6.86x kernel speedups for sparse speculative verification.