PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning

TL;DR - PATS is an adaptive training scaffold for long-horizon LLM agents that turns recent rollout evidence into task-specific guidance, then removes redundant support as the policy improves. It boosts agent performance while reducing prompt-token usage.

• Converts rollout groups from the latest policy into evidence cards that guide subsequent attempts.
• Dynamically revises or removes guidance to prevent overreliance while preserving rollout diversity.
• Uses standard RLVR with environmental rewards; the scaffold is discarded at deployment.
• Improves ALFWorld and WebShop results by up to 18.6% and uses 32.1% fewer prompt tokens across seven search-augmented QA benchmarks than the baseline.