Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents
TL;DR - This paper analyzes how reusable skills can cause LLM-agent failures and higher execution costs. Across two benchmarks, it identifies 307 skill-induced regressions and introduces SkillTriage for evidence-based failure attribution.
- The study finds 125 functional failures and 182 efficiency regressions.
- Seemingly relevant skills can cause agents to implement requirements incorrectly or omit them.
- Efficiency regressions are not attributable to prompt length alone.
- Excessive verification and heavy implementation pipelines account for 67 and 30 excessive-procedure cases, respectively.