ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Ranking
Overall
68
Content
80
Popularity
40
Observed public metrics from 1 member.
Merged summary
TL;DR - ResKV is a fixed-budget KV cache compression method that splits the budget into an exact main cache plus a compact "residual" cache reconstructing the attention mass of evicted tokens, improving long-context inference quality at equal memory cost.
- Frames information lost by eviction as residual statistics in both the numerator and denominator of softmax attention, rather than as discarded tokens.
- Residual entries join the same softmax normalization as main-cache tokens, restoring both numerator and denominator mass instead of applying a post-hoc correction.
- Allocation is set per layer and per KV head via a construction-time validation proxy, with a decode-time dynamic gate adjusting residual contribution per query.
- Reported broad gains on LongBench and RULER across query-aware/agnostic settings, multiple backbones, budgets, and baselines, while preserving peak memory and long-context decode throughput.
Sources (1)
ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - ResKV is a fixed-budget KV cache compression method that splits the budget into an exact main cache plus a compact "residual" cache reconstructing the attention mass of evicted tokens, improving long-context inference quality at equal memory cost.
- Frames information lost by eviction as residual statistics in both the numerator and denominator of softmax attention, rather than as discarded tokens.
- Residual entries join the same softmax normalization as main-cache tokens, restoring both numerator and denominator mass instead of applying a post-hoc correction.
- Allocation is set per layer and per KV head via a construction-time validation proxy, with a decode-time dynamic gate adjusting residual contribution per query.
- Reported broad gains on LongBench and RULER across query-aware/agnostic settings, multiple backbones, budgets, and baselines, while preserving peak memory and long-context decode throughput.