SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

TL;DR - SkillGate trains long-horizon agents to select the right procedural skill by separating selection credit from execution credit. This raises a 9B policy’s trial success from 40.8% to 53.2% across five agentic benchmarks while reducing misleading skill exposure.

• Identifies “selector credit starvation,” where sequence-level rewards give skill-selection tokens vanishing and increasingly wrong-signed credit as trajectories lengthen.
• Uses disjoint credit channels: outcome rewards train execution tokens, while an action-local advantage trains only skill-naming tokens.
• Evaluates selection from a 16-skill candidate slate across five benchmarks.
• Outperforms outcome-reward-only training at the same budget, cuts exposure to misleading candidates by two thirds, and reads fewer skills.