ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

TL;DR - ContextPilot is a reinforcement-learning framework that teaches LLM agents to proactively manage their working context during long-horizon tasks. It improves performance on long-context QA and deep-search benchmarks while maintaining a more compact context.

• Expands context-management tools beyond search, deletion, and summarization to include global planning, long-term memory, and soft context offloading.
• Identifies consequential editing decisions using changes in context and output entropy, then focuses trajectory branching on those actions.
• Uses branched trajectories to estimate action-level advantages, providing finer-grained credit assignment than applying one final reward to every context edit.
• Consistently outperforms existing baselines across multiple base models and benchmarks, according to the reported experiments.