‹ back to 2026-08-20

SPADE: Self-Play in Adaptive Synthetic Executable Environments

arXiv cs.CL LLM Agents Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao, Zichen Liu, Junsu Kim, Zijian Zhou, Seungone Kim, Tongzheng Ren, Mickel Liu, Hanfei Yu, Zhaorun Chen, Weiyan Shi, Paul Pu Liang, Luke Zettlemoyer, Yejin Choi, Natasha Jaques 2026-08-19

TL;DR - SPADE is a self-play reinforcement-learning framework in which one LLM alternates between designing executable, long-horizon environments and learning to solve them. Adaptive environment generation targets the learner’s capability frontier and outperforms fixed-environment training across reasoning, tool-use, and game evaluations.

The Environment Designer produces stateful OpenAI Gym-style environments with transitions, rewards, and verification code.
A hint-based regret signal steers generation toward tasks that are challenging but feasible for the Reasoning Agent.
Document grounding and accumulated environment memory are identified as critical design components.
At 30B parameters, SPADE improves over the strongest fixed-environment baseline by 5.3 points across eight held-out benchmarks, 5.7 on BFCL-v4 multi-turn, and 13.9 on ACEBench-Agent.

view merged work →