‹ back to 2026-09-04

Environment Evolution for Terminal Agents

arXiv cs.AI LLM Agents Zhiyuan Fan, Tinghao Yu, Yuanjun Cai, Jiang Zhou, Jiangtao Guan, Jincheng Liu, Yun Yang, Dingxin Hu, Zhuo Han, Xing Wu, Feng Zhang, Lilin Wang 2026-09-03

TL;DR - This paper introduces off-policy environment evolution, which progressively raises task difficulty to provide sustained training signals for terminal agents. Long-horizon RL using these environments improved two Qwen3.6 models on Terminal-Bench 2.1 by 14.4 and 18.0 percentage points.

Derives three environment-evolution directions from the multi-turn learning objective.
Uses a loop-engineered multi-agent harness to evolve environments and schedule successive generations during training.
Rollouts with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol consistently found the evolved environments more difficult.
Unlike on-policy co-evolution, the method is designed to keep generating useful challenges as the trained model becomes stronger.

view merged work →