TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy

TL;DR - TwinKV is a training- and attention-free repair pass that improves existing KV-cache eviction policies by swapping retained redundant tokens for evicted tokens lacking surviving duplicates. It can improve long-context inference under fixed memory budgets, though gains vary by model, policy, compression ratio, and task.

• A leave-one-out analysis found essentially no correlation between attention magnitude and a token’s causal contribution to an answer (Spearman ρ = -0.004).
• TwinKV detects pairwise key redundancy, replacing redundant retained “donors” with evicted “orphans” while preserving the original policy’s cache budget and scoring rule.
• Across LongBench, LooGLE, and RULER, it improved many configurations for several eviction policies; benefits were generally smaller at looser compression and for a strong adaptive baseline near its performance ceiling.
• Few-shot classification exemplars were a consistent failure case where TwinKV did not help either evaluated model.