‹ back to 2026-07-21

C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference

Research Efficiency & Systems

Merged summary

TL;DR - C²KV compresses and composes reusable, non-prefix KV caches to reduce long-context LLM inference costs. It reports up to 17× inference speedup while preserving generation quality.

Learns position-agnostic KV representations that can be reused and concatenated across contexts.
Uses a lightweight sidecar Extractor with learnable compression tokens and structured attention.
Keeps the base model frozen, avoiding architectural modifications.
Jointly trains compression and concatenation to prevent the accuracy degradation seen with naive combinations.

Sources (1)

C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference

arXiv cs.CL Chuheng Du, Junyi Chen, Hanlin Tang, Kan Liu, Tao Lan, Lin Qu, Chaoyue Niu, Shengzhong Liu, Guihai Chen, Fan Wu 2026-07-20 arXiv:2607.17715 doi:10.1145/3770855.3817715

TL;DR - C²KV compresses and composes reusable, non-prefix KV caches to reduce long-context LLM inference costs. It reports up to 17× inference speedup while preserving generation quality.

Learns position-agnostic KV representations that can be reused and concatenated across contexts.
Uses a lightweight sidecar Extractor with learnable compression tokens and structured attention.
Keeps the base model frozen, avoiding architectural modifications.
Jointly trains compression and concatenation to prevent the accuracy degradation seen with naive combinations.

item →