Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember

TL;DR - SESA (Self-Evolving Skill-Augmented Agent) couples self-play task generation with an evolving procedural skill memory, so a challenger's problem distribution and the solver's skill bank co-evolve instead of the curriculum being stateless. It matters because it shows external memory can reshape policy learning itself, not just serve as an inference-time add-on.

• Architecture: a challenger poses problems while a separately parameterized solver retrieves skills; informative failures are distilled into reusable skills written back to memory, and updated memory shifts solver success, challenger reward, and future problem distribution — a bidirectional loop.
• Because retrieved skills shape on-policy training trajectories, gains are absorbed into model parameters, allowing memory-free deployment (SESA-Off) with optional inference-time retrieval.
• Results across seven open-domain and multi-hop QA benchmarks: +1.2–3.2 average accuracy points over SSP across backbones, +0.9 over the skill-augmented SkillRL baseline under a unified protocol.
• On Qwen3, SESA-Off keeps 1.8–2.2 points over SSP without the memory bank, and re-adding the final skill bank contributes another 0.5–1.0 points; code is released publicly.