Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?

TL;DR - Skill-Use is a benchmark testing whether LLM agents can autonomously recognize, retrieve, and follow "skills" (structured procedure documents) under progressive disclosure; it finds current agents unreliable, with the best configuration scoring only 0.613 SU.

• Decomposes skill use into three facets: Trigger (invokes the relevant skill), Compliance (follows the prescribed procedure), and Boundary (avoids forbidden operations); the combined SU score credits execution only after triggering.
• Benchmark scope: 79 real skills paired with 177 executable tasks across nine domains, grounded in real files, run in isolated Docker sandboxes and graded by a trajectory-based rubric.
• Evaluated eight LLMs under two agent harnesses; triggering and procedural compliance emerged as independent bottlenecks rather than a single failure mode.
• Both absolute scores and model rankings shifted with the harness, implying skill use is a harness-conditioned capability, not a fixed model property — so harness choice must be reported in agent evaluations.