SemPIC: Learning Semantic Position-Independent KV Caches

TL;DR - SemPIC learns position-independent KV caches for reusable documents, improving long-context retrieval when instructions, histories, or document order change. It approaches full-recompute accuracy while preserving the standard cache-hit decoding path.

• A LoRA-enabled Writer compiles per-layer document KVs via behavioral distillation, while the pretrained Reader remains unchanged.
• Adaptation occurs only during offline cache construction, retaining the standard KV-cache interface.
• KV Gradient Checkpointing reduces peak training memory while preserving gradients through cached KVs.
• Across three models and four tasks, mean micro-F1 improves from 0.53 with KV Packet to 0.60, versus 0.62 for Full Recompute.