Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Ranking
Overall
75
Content
90
Popularity
39
Observed public metrics from 1 member.
Merged summary
TL;DR - Identical token prefixes can yield different continuations depending on whether inference reuses a live KV cache or rebuilds it via BF16 prefill. The divergence disappears with FP32, while cache-transplant experiments show that boundary KV state causally determines the divergent trajectory.
- BF16 live-cache versus fresh-prefill runs differed on 166 of 200 suffixes and 20 correctness labels, despite only a one-point accuracy gap.
- FP32 replay produced no decoded disagreements across 200 examples.
- Transplanting all 48 KV layers made continuations follow the donor cache in 24/24 selected cases and 43/43 later replications.
- Exact-token replay can be reproducible without faithfully reconstructing the original decoder state.
Sources (1)
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Identical token prefixes can yield different continuations depending on whether inference reuses a live KV cache or rebuilds it via BF16 prefill. The divergence disappears with FP32, while cache-transplant experiments show that boundary KV state causally determines the divergent trajectory.
- BF16 live-cache versus fresh-prefill runs differed on 166 of 200 suffixes and 20 correctness labels, despite only a one-point accuracy gap.
- FP32 replay produced no decoded disagreements across 200 examples.
- Transplanting all 48 KV layers made continuations follow the donor cache in 24/24 selected cases and 43/43 later replications.
- Exact-token replay can be reproducible without faithfully reconstructing the original decoder state.