Negative Self-Distillation: Learning to Reason by Avoiding Flaws

TL;DR - Negative Self-Distillation (NSD) improves LLM reasoning by teaching models to diverge from self-generated flawed reasoning rather than imitate privileged, artificially confident solutions. This preserves exploratory and self-corrective behavior that conventional on-policy self-distillation may suppress.

• NSD requires neither ground-truth answers nor external supervision; the model generates a question-specific negative persona, such as a “careless reasoner.”
• A dynamic gating mechanism isolates reasoning-critical tokens so training targets behavioral flaws without degrading foundational language abilities.
• The approach addresses the confounding of flawed reasoning with ordinary linguistic tokens that makes naive unlearning objectives risky.
• NSD reportedly outperforms on-policy self-distillation and other label-free, self-bootstrapping reinforcement-learning baselines.