ICML 2026 | 一步错不再全盘输，浙大BEACON让长程Agent成功率近翻倍

TL;DR - BEACON is an ICML 2026 framework for training long-horizon language agents by assigning credit at environment-defined milestones rather than relying only on sparse terminal rewards. On long ALFWorld tasks, it raised success from 53.5% with GRPO to 92.9% while substantially improving sample utilization.

• BEACON partitions trajectories at verified subgoals, applies temporally decayed rewards within each segment, and combines trajectory-level and segment-level advantage estimates.
• It requires no additional reward model; milestones come from environment states or built-in subgoal signals in ALFWorld, WebShop, and ScienceWorld.
• Effective sample utilization increased from 23.7% to 82.0%, and the zero-advantage ratio fell from roughly 55% to 10%.
• Gains grew with task length, suggesting milestone-based credit assignment particularly addresses failures in long-horizon agent reinforcement learning.