‹ back to 2026-08-10

Bidirectional Context Self-Distillation for Reinforcement Learning of Skill-Based LLM Agents

arXiv cs.AI LLM Agents Tianjun Pan, Yuan Li, Hongda Wang, Linbo Jin, Mengfei Song, Lei Gao, Qiming Shi, Shaokang Fu, Jiarong Zhao, Chengyu Wang, Chengfu Huo 2026-08-10

TL;DR - An arXiv preprint proposing BCSD (Bidirectional Context Self-Distillation), an RL framework that trains LLM agents to better exploit external natural-language skills. It matters because task-level rewards alone give weak supervision on how well a policy actually uses provided guidance.

Core idea: evaluate each trajectory from two complementary skill-context views instead of a single privileged context — an augmented view adding higher-level "Meta-Skill" guidance, and a reduced view that prunes general guidance to expose task-specific skills.
The token-level signals from both views are combined to rescale the RL advantage, providing finer-grained supervision on skill utilization than task-level reward alone.
Reported results: strongest overall performance across model scales on ALFWorld and WebShop; ablations indicate the augmented and reduced views contribute complementarily.
Authors state code will be released for reproducibility; no specific numeric scores or baselines are given in the abstract.

view merged work →