KV Cache Compression Through the Lens of Transform Coding
Ranking
Overall
82
Content
100
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - Attention-Aware Transform Coding compresses LLM KV caches by optimizing bit allocation for attention-level distortion rather than cache reconstruction error. It achieves near-lossless accuracy at roughly 5.8Ă— compression across two instruction-tuned models and several benchmarks.
- Derives additive key and value distortion terms that factor across tokens and channels under a white-noise quantization model.
- Uses transform coding and reverse water-filling to allocate bits based on calibration data.
- Evaluated on Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct across long-context, reasoning, and knowledge benchmarks.
- Unlike the reported baselines, AATC avoids notable degradation across the tested settings.
Sources (1)
KV Cache Compression Through the Lens of Transform Coding
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Attention-Aware Transform Coding compresses LLM KV caches by optimizing bit allocation for attention-level distortion rather than cache reconstruction error. It achieves near-lossless accuracy at roughly 5.8Ă— compression across two instruction-tuned models and several benchmarks.
- Derives additive key and value distortion terms that factor across tokens and channels under a white-noise quantization model.
- Uses transform coding and reverse water-filling to allocate bits based on calibration data.
- Evaluated on Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct across long-context, reasoning, and knowledge benchmarks.
- Unlike the reported baselines, AATC avoids notable degradation across the tested settings.