Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

TL;DR - This paper introduces cross-model KV cache transfer, using learned mappings to reuse prefill caches when switching between related LLMs. The approach can substantially reduce model-handoff latency while often preserving most downstream accuracy.

• A closed-form, per-head ridge mapper predicts target-model KV states from selected source-model layers after removing RoPE from keys.
• Calibration uses 500 FineWeb-Edu sequences of 1,024 tokens each.
• Across six model pairs in three families, the linear method retained 73–98% of baseline accuracy on four pairs; an MLP improved failure cases by up to 37 percentage points on HellaSwag.
• Mapping was 2.7–25× faster than recomputing prefill and remained stable across multi-turn handoffs.