Where Should a Document Live: Context, Representations, or Parameters?
Ranking
Overall
82
Content
100
Popularity
39
Observed public metrics from 1 member.
Merged summary
TL;DR - This paper compares storing new knowledge in LLM context, KV-cache representations, or fine-tuned parameters. KV-based Cartridges deliver the strongest accuracy and retrieval performance but can cause substantial catastrophic forgetting.
- In oracle tests across five knowledge-intensive benchmarks, Cartridges outperform parametric methods by roughly 10 points at nearly every storage budget.
- KV Compaction remains competitive only at low compression, trailing parametric methods by 10 points beyond (50\times) compression.
- In multi-document retrieval, Cartridges alone match in-context learning, beating parametric methods by 29 points and Compaction by 15 points.
- Cartridges reduce control-benchmark performance by 6%, including a 13% decline on coding tasks.
Sources (1)
Where Should a Document Live: Context, Representations, or Parameters?
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - This paper compares storing new knowledge in LLM context, KV-cache representations, or fine-tuned parameters. KV-based Cartridges deliver the strongest accuracy and retrieval performance but can cause substantial catastrophic forgetting.
- In oracle tests across five knowledge-intensive benchmarks, Cartridges outperform parametric methods by roughly 10 points at nearly every storage budget.
- KV Compaction remains competitive only at low compression, trailing parametric methods by 10 points beyond (50\times) compression.
- In multi-document retrieval, Cartridges alone match in-context learning, beating parametric methods by 29 points and Compaction by 15 points.
- Cartridges reduce control-benchmark performance by 6%, including a 13% decline on coding tasks.