RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution

TL;DR - RedEvoAgent is a black-box red-teaming agent that converts prior jailbreak trajectories into compact, interpretable attack skills that evolve with experience. It aims to expose dangerous tool use and persistent state changes in deployed LLM agents more efficiently and reliably.

• Profiles tool effectiveness and uses Deciding-Tool Attribution to determine which tools deserve credit for successful attacks.
• Applies a validation ratchet that retains skill updates only when they improve validation performance.
• Avoids the context overhead, retrieval bias, and limited interpretability associated with reusing full attack trajectories.
• Outperforms fixed and agentic baselines across multiple benchmarks, models, and execution harnesses, while improving tool efficiency and transferability.