TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Ranking
Overall
83
Content
100
Popularity
42
Observed public metrics from 1 member.
Merged summary
TL;DR - TwinKV is a training- and attention-free repair pass that improves existing KV-cache eviction policies by swapping retained redundant tokens for evicted tokens lacking surviving duplicates. It can improve long-context inference under fixed memory budgets, though gains vary by model, policy, compression ratio, and task.
- A leave-one-out analysis found essentially no correlation between attention magnitude and a token’s causal contribution to an answer (Spearman ρ = -0.004).
- TwinKV detects pairwise key redundancy, replacing redundant retained “donors” with evicted “orphans” while preserving the original policy’s cache budget and scoring rule.
- Across LongBench, LooGLE, and RULER, it improved many configurations for several eviction policies; benefits were generally smaller at looser compression and for a strong adaptive baseline near its performance ceiling.
- Few-shot classification exemplars were a consistent failure case where TwinKV did not help either evaluated model.
Sources (1)
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - TwinKV is a training- and attention-free repair pass that improves existing KV-cache eviction policies by swapping retained redundant tokens for evicted tokens lacking surviving duplicates. It can improve long-context inference under fixed memory budgets, though gains vary by model, policy, compression ratio, and task.
- A leave-one-out analysis found essentially no correlation between attention magnitude and a token’s causal contribution to an answer (Spearman ρ = -0.004).
- TwinKV detects pairwise key redundancy, replacing redundant retained “donors” with evicted “orphans” while preserving the original policy’s cache budget and scoring rule.
- Across LongBench, LooGLE, and RULER, it improved many configurations for several eviction policies; benefits were generally smaller at looser compression and for a strong adaptive baseline near its performance ceiling.
- Few-shot classification exemplars were a consistent failure case where TwinKV did not help either evaluated model.