Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

TL;DR - BIRD is a two-stage self-distillation method that compresses chain-of-thought reasoning by first fixing the student model's rollout distribution before on-policy training, yielding shorter reasoning traces without sacrificing accuracy. It matters because it targets the wasted computation in long CoT traces that inflates inference cost.

• Diagnoses an "initialization bottleneck" in on-policy self-distillation: since supervision only covers visited prefixes, a verbose base model places KL loss on noisy/redundant/off-track contexts, limiting a concise teacher to local corrections.
• Stage 1 samples concise, answer-correct traces from the base model under a brevity instruction, then does a lightweight prompt-switch SFT (traces generated with the brevity prompt but learned under the original task prompt) to make conciseness default behavior.
• Stage 2 applies on-policy reverse-KL distillation with a concise self-teacher on the resulting cleaner prefixes.
• On Qwen3-8B, improves MATH-500 accuracy 86.2%→92.0% while cutting average response length 3,099→1,115 tokens; reports a better accuracy-efficiency trade-off than prompting and cold-start distillation on MATH-500/AIME across Qwen3 models.