‹ back to 2026-09-16

Where Should a Document Live: Context, Representations, or Parameters?

arXiv cs.CL LLMs & Foundation Models Nathanaël Carraz Rakotonirina, Momchil Hardalov, Gonzalo Iglesias, Adrià de Gispert 2026-09-15

TL;DR - This paper compares storing new knowledge in LLM context, KV-cache representations, or fine-tuned parameters. KV-based Cartridges deliver the strongest accuracy and retrieval performance but can cause substantial catastrophic forgetting.

In oracle tests across five knowledge-intensive benchmarks, Cartridges outperform parametric methods by roughly 10 points at nearly every storage budget.
KV Compaction remains competitive only at low compression, trailing parametric methods by 10 points beyond (50\times) compression.
In multi-document retrieval, Cartridges alone match in-context learning, beating parametric methods by 29 points and Compaction by 15 points.
Cartridges reduce control-benchmark performance by 6%, including a 13% decline on coding tasks.

view merged work →