Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
Ranking
Overall
81
Content
100
Popularity
38
Observed public metrics from 1 member.
Merged summary
TL;DR - This paper introduces cross-model KV cache transfer, using learned mappings to reuse prefill caches when switching between related LLMs. The approach can substantially reduce model-handoff latency while often preserving most downstream accuracy.
- A closed-form, per-head ridge mapper predicts target-model KV states from selected source-model layers after removing RoPE from keys.
- Calibration uses 500 FineWeb-Edu sequences of 1,024 tokens each.
- Across six model pairs in three families, the linear method retained 73–98% of baseline accuracy on four pairs; an MLP improved failure cases by up to 37 percentage points on HellaSwag.
- Mapping was 2.7–25× faster than recomputing prefill and remained stable across multi-turn handoffs.
Sources (1)
Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
Public signals
Hugging Face upvotes 1
TL;DR - This paper introduces cross-model KV cache transfer, using learned mappings to reuse prefill caches when switching between related LLMs. The approach can substantially reduce model-handoff latency while often preserving most downstream accuracy.
- A closed-form, per-head ridge mapper predicts target-model KV states from selected source-model layers after removing RoPE from keys.
- Calibration uses 500 FineWeb-Edu sequences of 1,024 tokens each.
- Across six model pairs in three families, the linear method retained 73–98% of baseline accuracy on four pairs; an MLP improved failure cases by up to 37 percentage points on HellaSwag.
- Mapping was 2.7–25× faster than recomputing prefill and remained stable across multi-turn handoffs.