SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving

TL;DR - SkillGym converts human-authored agent workflows into verifiable training environments, enabling LLMs to internalize reusable problem-solving skills rather than rely on inference-time instructions. Fine-tuning on verified trajectories substantially improved a 35B model across agent and terminal benchmarks.

• The released dataset includes 2,756 environments across 12 categories and 8,364 successful multi-model trajectories, averaging 49 tool calls each.
• Tasks use code-based outcome checkers and contrastive executions to verify both correctness and dependence on the target skill.
• Supervised fine-tuning improved Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2 and 19.10 percentage points on Terminal-Bench 2.1.
• The resulting SkillGym-Agent reached 51.47% on skill-assisted SkillsBench and retained gains without external skills, suggesting internalized procedural competence.