SemPIC: Learning Semantic Position-Independent KV Caches
Ranking
Overall
78
Content
95
Popularity
39
Observed public metrics from 1 member.
Merged summary
TL;DR - SemPIC learns position-independent KV caches for reusable documents, improving long-context retrieval when instructions, histories, or document order change. It approaches full-recompute accuracy while preserving the standard cache-hit decoding path.
- A LoRA-enabled Writer compiles per-layer document KVs via behavioral distillation, while the pretrained Reader remains unchanged.
- Adaptation occurs only during offline cache construction, retaining the standard KV-cache interface.
- KV Gradient Checkpointing reduces peak training memory while preserving gradients through cached KVs.
- Across three models and four tasks, mean micro-F1 improves from 0.53 with KV Packet to 0.60, versus 0.62 for Full Recompute.
Sources (1)
SemPIC: Learning Semantic Position-Independent KV Caches
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - SemPIC learns position-independent KV caches for reusable documents, improving long-context retrieval when instructions, histories, or document order change. It approaches full-recompute accuracy while preserving the standard cache-hit decoding path.
- A LoRA-enabled Writer compiles per-layer document KVs via behavioral distillation, while the pretrained Reader remains unchanged.
- Adaptation occurs only during offline cache construction, retaining the standard KV-cache interface.
- KV Gradient Checkpointing reduces peak training memory while preserving gradients through cached KVs.
- Across three models and four tasks, mean micro-F1 improves from 0.53 with KV Packet to 0.60, versus 0.62 for Full Recompute.