CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

TL;DR - CoinRAG is an arXiv preprint that speeds up long-context RAG by reusing precomputed KV caches at the level of fine-grained "information nuggets" instead of whole retrieved chunks, cutting prefill cost while improving answer quality. It matters because chunk-level KV reuse — the current efficiency trick for RAG — still feeds models redundant, noisy context.

• Motivation: existing chunk-level KV cache reuse avoids re-encoding long retrieved contexts, but coarse chunks carry substantial redundancy and noise.
• Method: a two-stage retrieval identifies query-relevant semantic units ("nuggets") inside retrieved chunks, then compositionally assembles their sliced, offline-computed KV representations together with a chunk-level context into a learned contextual representation.
• Framing: the goal is an improved accuracy/latency Pareto frontier under low prefill-latency constraints, rather than accuracy alone.
• Results: on LongBench multi-hop QA, CoinRAG reports a new Pareto frontier, reduced operational cost, and an average 5.3% relative F1 gain over baselines under a standard fast-prefill latency budget.