Contextual Information Policy Optimization for Search Agents

TL;DR - CIPO is a reinforcement learning framework for LLM search agents that rewards reasoning steps actually grounded in retrieved evidence, rather than only final-answer correctness, to counter "prior-driven reasoning" where agents guess from parametric memory and use retrieval merely as confirmation.

• Diagnoses a reward misalignment in existing search-agent RL: outcome- or progress-only rewards never check whether post-retrieval actions are grounded in the retrieved evidence, encouraging confirmation bias and inefficient evidence use.
• Assigns dense, turn-level credit to reasoning actions influenced by retrieved information, combined with a global outcome reward to preserve answer correctness.
• Requires no human process annotations and no separate reward model, making the evidence-use signal cheap to obtain relative to process-supervision approaches.
• Evaluated on seven in-domain and out-of-domain benchmarks; authors report reduced prevalence of prior-driven reasoning and strong performance on most tasks (no specific numbers given in the provided abstract).