EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - EnvACE is an agentic RL method that trains LLM tool-use agents without external environments: the policy alternates between emitting a tool call and "rehearsing" the environment response itself, internalizing environment dynamics as a built-in world model. It matters because it removes the costly environment construction/verification bottleneck in long-horizon agent training.
- The policy plays both actor and environment: it generates a tool call, then produces the induced response, and conditions later decisions on that rehearsed response; both roles are jointly optimized end-to-end with task-success rewards.
- Evaluated on BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, it reports strong, transferable results that outperform environment-scaling baselines overall.
- Controlled studies indicate world rehearsal improves policy learning consistently across model scales.
- At inference, the internalized world model supports private rehearsal before committing to an execution, giving further gains under a moderate rehearsal budget with no extra external interaction; code is released at github.com/Within-yao/EnvACE.
Sources (1)
EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
TL;DR - EnvACE is an agentic RL method that trains LLM tool-use agents without external environments: the policy alternates between emitting a tool call and "rehearsing" the environment response itself, internalizing environment dynamics as a built-in world model. It matters because it removes the costly environment construction/verification bottleneck in long-horizon agent training.
- The policy plays both actor and environment: it generates a tool call, then produces the induced response, and conditions later decisions on that rehearsed response; both roles are jointly optimized end-to-end with task-success rewards.
- Evaluated on BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, it reports strong, transferable results that outperform environment-scaling baselines overall.
- Controlled studies indicate world rehearsal improves policy learning consistently across model scales.
- At inference, the internalized world model supports private rehearsal before committing to an execution, giving further gains under a moderate rehearsal budget with no extra external interaction; code is released at github.com/Within-yao/EnvACE.