What It Costs to Compose, Rebuild, and Correct Precomputed Memory
Ranking
Overall
81
Content
100
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - This paper evaluates when reusable precomputed LLM memory remains correct and how much it costs to update. It finds that composing memories from separate parts and applying real-time corrections can be unreliable, while rebuilding often costs a substantial fraction of the original preparation.
- Experiments use Llama-3.1-8B-Instruct with saved key-value caches and trained cache compressions.
- Precomputed memory quality degrades when independently prepared components are assembled.
- Keeping memory current generally requires rebuilding it at the cadence of changes to its source material.
- Warm rebuilds and carefully phrased side-channel updates show promise, with the latter serving as an interim measure between rebuilds.
Sources (1)
What It Costs to Compose, Rebuild, and Correct Precomputed Memory
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - This paper evaluates when reusable precomputed LLM memory remains correct and how much it costs to update. It finds that composing memories from separate parts and applying real-time corrections can be unreliable, while rebuilding often costs a substantial fraction of the original preparation.
- Experiments use Llama-3.1-8B-Instruct with saved key-value caches and trained cache compressions.
- Precomputed memory quality degrades when independently prepared components are assembled.
- Keeping memory current generally requires rebuilding it at the cadence of changes to its source material.
- Warm rebuilds and carefully phrased side-channel updates show promise, with the latter serving as an interim measure between rebuilds.