KV Cache Compression Through the Lens of Transform Coding

TL;DR - Attention-Aware Transform Coding compresses LLM KV caches by optimizing bit allocation for attention-level distortion rather than cache reconstruction error. It achieves near-lossless accuracy at roughly 5.8× compression across two instruction-tuned models and several benchmarks.

• Derives additive key and value distortion terms that factor across tokens and channels under a white-noise quantization model.
• Uses transform coding and reverse water-filling to allocate bits based on calibration data.
• Evaluated on Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct across long-context, reasoning, and knowledge benchmarks.
• Unlike the reported baselines, AATC avoids notable degradation across the tested settings.