CAFE: Self-Improving Search Agents Need Co-Evolving Feedback
Ranking
Overall
83
Content
95
Popularity
56
Observed public metrics from 1 member.
Merged summary
TL;DR - CAFE jointly improves an agentic search policy and its feedback critic by training them in alternating, coupled roles. Across seven in-domain and six out-of-domain benchmarks, this approach outperformed evaluated RL-based search agents on average and reduced answer-level hallucinations.
- A shared-parameter model alternates between acting as the search agent and providing corrective, in-trajectory feedback.
- Online RL rewards useful feedback requests using a call-versus-skip success gap and reshapes token-level advantages around feedback.
- Offline preference optimization trains feedback from matched successful and unsuccessful rollout trajectories.
- Ablations found that improving only the agent or critic eventually plateaued, while alternating updates continued to improve performance.
Sources (1)
CAFE: Self-Improving Search Agents Need Co-Evolving Feedback
Public signals
Hugging Face upvotes 6 · Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - CAFE jointly improves an agentic search policy and its feedback critic by training them in alternating, coupled roles. Across seven in-domain and six out-of-domain benchmarks, this approach outperformed evaluated RL-based search agents on average and reduced answer-level hallucinations.
- A shared-parameter model alternates between acting as the search agent and providing corrective, in-trajectory feedback.
- Online RL rewards useful feedback requests using a call-versus-skip success gap and reshapes token-level advantages around feedback.
- Offline preference optimization trains feedback from matched successful and unsuccessful rollout trajectories.
- Ablations found that improving only the agent or critic eventually plateaued, while alternating updates continued to improve performance.