KVGuard: A KV Cache Safety Checker for LLM Serving Systems

Summary

KVGuard is a memory-safety checker for the KV cache in LLM serving systems, which use PagedAttention to map requests onto fixed-size physical blocks through a CPU-built block table. It performs five safety checks across the PagedAttention workflow, using compact per-block shadow metadata, and catches the boundary where a violation is introduced. A prototype on vLLM and SGLang covers all 29 historical KV cache safety bugs from the two systems' patch histories, finds 8 previously unknown violations with no false positives, adds 3.0% and 7.4% geomean TPOT overhead on Qwen 2.5 14B, and uses at most 0.11% of KV cache memory.