Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
Ranking
Overall
77
Content
95
Popularity
34
Observed public metrics from 1 member.
Merged summary
TL;DR - This study finds that on-policy distillation improves LLM reasoning mainly by making successful outputs more likely with small sampling budgets, rather than expanding the model’s ultimate reasoning capabilities.
- OPD models sustain better avg@K across sampling budgets.
- As K increases, pre-OPD models gradually overtake OPD models on pass@K.
- Training shifts performance toward stronger small-K results at the expense of the large-K capability boundary.
- At pass@1024, OPD makes more previously solvable problems unsolvable than it makes previously unsolvable problems solvable.
Sources (1)
Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - This study finds that on-policy distillation improves LLM reasoning mainly by making successful outputs more likely with small sampling budgets, rather than expanding the model’s ultimate reasoning capabilities.
- OPD models sustain better avg@K across sampling budgets.
- As K increases, pre-OPD models gradually overtake OPD models on pass@K.
- Training shifts performance toward stronger small-K results at the expense of the large-K capability boundary.
- At pass@1024, OPD makes more previously solvable problems unsolvable than it makes previously unsolvable problems solvable.