Coupled Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - Coupled Calibration and Learning (CCL) is an LLM distillation method that jointly calibrates a biased teacher and trains a student under target-domain covariate shift, without target-domain reward feedback. The authors theoretically show that CCL can recover the optimal student within its policy class, whereas direct teacher matching may retain persistent error.
- CCL uses source-domain reward feedback to calibrate the teacher, then trains the student on target questions through token-level branching.
- Teacher calibration and student learning form an iterative feedback loop, with each updated student informing subsequent calibration.
- In an autoregressive policy framework, the student’s expected average KL divergence to an oracle student converges to zero at a polynomial rate.
- Regularized direct matching can remain bounded away from the oracle even when the teacher outperforms every student policy on regularized target reward.
Sources (1)
Coupled Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback
TL;DR - Coupled Calibration and Learning (CCL) is an LLM distillation method that jointly calibrates a biased teacher and trains a student under target-domain covariate shift, without target-domain reward feedback. The authors theoretically show that CCL can recover the optimal student within its policy class, whereas direct teacher matching may retain persistent error.
- CCL uses source-domain reward feedback to calibrate the teacher, then trains the student on target questions through token-level branching.
- Teacher calibration and student learning form an iterative feedback loop, with each updated student informing subsequent calibration.
- In an autoregressive policy framework, the student’s expected average KL divergence to an oracle student converges to zero at a polynomial rate.
- Regularized direct matching can remain bounded away from the oracle even when the teacher outperforms every student policy on regularized target reward.