Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search

TL;DR - This arXiv paper shows that a document's static retrieval usefulness (does it help a reader answer a question directly?) barely predicts its causal value inside a multi-step LLM search agent, meaning current retrieval training objectives may be mis-aligned with what agentic systems actually need.

• Using a ReAct-style agent over 1,000 HotpotQA dev questions, the authors compute a Counterfactual Trajectory Utility (CTU) by deleting each read document and replaying the rest of the trajectory, scoring deltas in final answer quality, next-query retrieval quality, and turn count.
• Across 23,322 document observations, CTU and Static RAG Utility (SRU) are nearly independent (Spearman ρ = −0.026); ~1/3 of read documents are causally load-bearing yet look useless to a static reader — dubbed "bridge documents" (27.2% under a BM25 + cross-encoder proxy).
• Mechanism: bridge documents supply discriminative entities that redirect the search — entities distinguishing relevant from non-relevant candidates appear in the agent's next query 4.02× more often (6.1% vs 1.5%, n = 227,139).
• Takeaway: static relevance and causal usefulness are distinct quantities in agentic retrieval, so optimizing static utility does not deliver agentic utility.