Contextual Information Policy Optimization for Search Agents
Ranking
Overall
64
Content
75
Popularity
39
Observed public metrics from 1 member.
Merged summary
TL;DR - CIPO is a reinforcement learning framework for LLM search agents that rewards reasoning steps actually grounded in retrieved evidence, rather than only final-answer correctness, to counter "prior-driven reasoning" where agents guess from parametric memory and use retrieval merely as confirmation.
- Diagnoses a reward misalignment in existing search-agent RL: outcome- or progress-only rewards never check whether post-retrieval actions are grounded in the retrieved evidence, encouraging confirmation bias and inefficient evidence use.
- Assigns dense, turn-level credit to reasoning actions influenced by retrieved information, combined with a global outcome reward to preserve answer correctness.
- Requires no human process annotations and no separate reward model, making the evidence-use signal cheap to obtain relative to process-supervision approaches.
- Evaluated on seven in-domain and out-of-domain benchmarks; authors report reduced prevalence of prior-driven reasoning and strong performance on most tasks (no specific numbers given in the provided abstract).
Sources (1)
Contextual Information Policy Optimization for Search Agents
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - CIPO is a reinforcement learning framework for LLM search agents that rewards reasoning steps actually grounded in retrieved evidence, rather than only final-answer correctness, to counter "prior-driven reasoning" where agents guess from parametric memory and use retrieval merely as confirmation.
- Diagnoses a reward misalignment in existing search-agent RL: outcome- or progress-only rewards never check whether post-retrieval actions are grounded in the retrieved evidence, encouraging confirmation bias and inefficient evidence use.
- Assigns dense, turn-level credit to reasoning actions influenced by retrieved information, combined with a global outcome reward to preserve answer correctness.
- Requires no human process annotations and no separate reward model, making the evidence-use signal cheap to obtain relative to process-supervision approaches.
- Evaluated on seven in-domain and out-of-domain benchmarks; authors report reduced prevalence of prior-driven reasoning and strong performance on most tasks (no specific numbers given in the provided abstract).