长程自主 Agent 怎么训？盘点近期 Agentic RL 代表性工作

TL;DR - This technical survey reviews recent Agentic RL systems for training long-horizon, tool-using LLM agents and distills eight practical design principles. It highlights advances in scalable rollouts, modular environments, curriculum learning, synthetic tasks, and collapse prevention.

• ToRL learns tool use directly through RL, while AgentGym-RL progressively increases interaction budgets to stabilize long-horizon training.
• Agent-R1 uses step-level trajectories and action masking; its experiments show context-management strategy can materially affect performance.
• AgentRL decouples asynchronous rollouts from training, and AutoForge uses LLMs to synthesize stateful environments and tasks.
• RAGEN identifies the Echo Trap and template collapse, using reward-variance filtering and entropy-preserving optimization to improve stability.