Where the Savings Go: The Limits of Hiding Cross-Tenant Cache Hits in LLM Serving
- Published
- Record updated
Summary
This research examines cross-tenant prefix-cache timing leakage in LLM serving engines, where a cache hit reveals whether other users sent a given text. Step-based schedulers pass the saved time to every request queued behind the hit, so an attacker can detect cached texts of 256 tokens or more with advantage at least 0.98 using one probe on one vLLM instance at up to half capacity. Hiding only the hit request falls to one probe, and the authors prove a rate-limited tenant's gain from content written by k other accounts is at most of order k times the allowed leakage.
Mitigation
Exact hiding is attainable in principle on one instance, but mechanisms that only pause or hold work know skipped work's duration only approximately, so under saturated prefill they must pad every step that may hold a hit, at a price that grows with time rather than with hits.
Related items
- MediumAmazon Uncovers AI Agent Campaign Targeting LiteLLM, Dify, and Other Apps for API KeysSimilar attack · The Hacker News
- InfoRisk-certified prompt optimization for LLM-as-a-judge guardrailsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- MediumCursor confirms security breach led to malicious Claude Code extensionSimilar attack · BleepingComputer
- MediumAmazon Threat Intelligence observes threat actors running cybercrime campaigns with ARTEX AI agentsSimilar attack · AWS Security Blog
- InfoForget the Lineage: Forward-Secure, Lineage-Complete Cryptographic Erasure for LLM-Agent MemorySimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)