When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Ranking
Overall
71
Content
85
Popularity
39
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv paper showing that self-evolving LLM agents suffer a "capability contamination" phase transition, where accumulating distilled skills past a critical pool size degrades performance irreversibly, and proposes a pre-commit verifier gate (VaG) to admit only safe skills.
- Contamination is structural: a defective skill entering the decision context becomes reference material for later skill distillation, forming cross-round contamination chains; post-hoc removal of the culprit skill recovers only a small fraction of lost performance.
- VaG uses a progressive trust hierarchy of three heterogeneous critics — structural validity, behavioral harmlessness, semantic consistency — filtering each skill individually, plus marginal-gain subset selection at the top tier to remove combinatorial contamination before runtime.
- On Terminal-Bench 2, unconditional accumulation peaks then degrades, while VaG improves every round to 72% pass@1 with a roughly 5x smaller skill pool.
- The frozen VaG skill pool transfers positively to four other backbones and a second benchmark without re-evolution; ablations show the three critics are complementary and intercept largely disjoint classes of harmful skills.
Sources (1)
When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - An arXiv paper showing that self-evolving LLM agents suffer a "capability contamination" phase transition, where accumulating distilled skills past a critical pool size degrades performance irreversibly, and proposes a pre-commit verifier gate (VaG) to admit only safe skills.
- Contamination is structural: a defective skill entering the decision context becomes reference material for later skill distillation, forming cross-round contamination chains; post-hoc removal of the culprit skill recovers only a small fraction of lost performance.
- VaG uses a progressive trust hierarchy of three heterogeneous critics — structural validity, behavioral harmlessness, semantic consistency — filtering each skill individually, plus marginal-gain subset selection at the top tier to remove combinatorial contamination before runtime.
- On Terminal-Bench 2, unconditional accumulation peaks then degrades, while VaG improves every round to 72% pass@1 with a roughly 5x smaller skill pool.
- The frozen VaG skill pool transfers positively to four other backbones and a second benchmark without re-evolution; ablations show the three critics are complementary and intercept largely disjoint classes of harmful skills.