LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding
Ranking
Overall
82
Content
100
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - LOCKS accelerates long-context LLM decoding by using compact, page-local spectral summaries to select which KV-cache pages deserve attention. It matches FullKV aggregate quality at 100K+ context while attending to about 2% of tokens and halves per-token latency at 1M tokens.
- Each KV-cache page gets a compact summary roughly one-tenth its size, preserving page-specific attention directions.
- Page selection reads summaries rather than candidate keys or values, then applies attention only to the highest-mass pages.
- LOCKS outperforms baseline selectors on retrieval-dense and long-form reasoning benchmarks, where shared-basis approaches can degrade sharply.
- It integrates with unmodified vLLM as a drop-in plugin and supports batched decoding with full CUDA graphs.
Sources (1)
LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - LOCKS accelerates long-context LLM decoding by using compact, page-local spectral summaries to select which KV-cache pages deserve attention. It matches FullKV aggregate quality at 100K+ context while attending to about 2% of tokens and halves per-token latency at 1M tokens.
- Each KV-cache page gets a compact summary roughly one-tenth its size, preserving page-specific attention directions.
- Page selection reads summaries rather than candidate keys or values, then applies attention only to the highest-mass pages.
- LOCKS outperforms baseline selectors on retrieval-dense and long-form reasoning benchmarks, where shared-basis approaches can degrade sharply.
- It integrates with unmodified vLLM as a drop-in plugin and supports batched decoding with full CUDA graphs.