Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv preprint arguing that chunk-and-embed top-k RAG is structurally unsound for table-heavy financial/regulatory documents, and proposing READ, an embedding-free agentic search loop over deterministic document operations. It matters because it reframes retrieval quality as an interface problem and yields replayable audit trails instead of opaque similarity scores.
- Diagnosis on a 780-page government financial report: 86.8% of content lines are table rows, near-identical figures collide in embedding space, and units sit a median of 13 lines above a figure — so chunk boundaries can cause lakh-vs-crore errors of two orders of magnitude.
- A steelman table-aware chunker fixes units but still leaves 27–30% of numeric chunks without a fiscal-year header at every chunk size tested.
- READ exposes three deterministic operations over the Model Context Protocol — normalized lexical search, structural navigation, and bounded span reads — making each trajectory an auditable trail.
- On 51 verified questions: READ 58.8% vs dense retrieval 15.7% (p_Holm = 2e-5), or 35.3% tuned (READ leads by 23.5 pts, p_Holm = 0.017); the same agent loop with a top-k tool reaches only 27.5%, locating the gain in the interface. BM25 is statistically indistinguishable from READ, so the result separates embedding-based from embedding-free retrieval, not agentic from lexical search.
Sources (1)
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
TL;DR - An arXiv preprint arguing that chunk-and-embed top-k RAG is structurally unsound for table-heavy financial/regulatory documents, and proposing READ, an embedding-free agentic search loop over deterministic document operations. It matters because it reframes retrieval quality as an interface problem and yields replayable audit trails instead of opaque similarity scores.
- Diagnosis on a 780-page government financial report: 86.8% of content lines are table rows, near-identical figures collide in embedding space, and units sit a median of 13 lines above a figure — so chunk boundaries can cause lakh-vs-crore errors of two orders of magnitude.
- A steelman table-aware chunker fixes units but still leaves 27–30% of numeric chunks without a fiscal-year header at every chunk size tested.
- READ exposes three deterministic operations over the Model Context Protocol — normalized lexical search, structural navigation, and bounded span reads — making each trajectory an auditable trail.
- On 51 verified questions: READ 58.8% vs dense retrieval 15.7% (p_Holm = 2e-5), or 35.3% tuned (READ leads by 23.5 pts, p_Holm = 0.017); the same agent loop with a top-k tool reaches only 27.5%, locating the gain in the interface. BM25 is statistically indistinguishable from READ, so the result separates embedding-based from embedding-free retrieval, not agentic from lexical search.