Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Ranking
Overall
84
Content
90
Popularity
71
Observed public metrics from 1 member.
Merged summary
TL;DR - Self-OPD is a teacher-free on-policy distillation framework that uses a flow matching model’s own stochastic exploration to generate step-wise supervision. It aims to reduce teacher-training costs and distribution mismatch while improving single- and multi-objective alignment.
- At each timestep, the method creates multiple stochastic SDE branches, rolls them out with an ODE sampler, and evaluates their rewards against a deterministic self-reference.
- A pull-push objective attracts the model toward high-advantage branches and repels it from low-advantage ones, with direction-aware attenuation and variance normalization.
- Multi-objective alignment combines normalized scores at the reward level to avoid direct gradient conflicts.
- Reported experiments show improvements over prior reinforcement-learning and OPD methods without task-specific teachers.
Sources (1)
Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Public signals
Hugging Face upvotes 64
TL;DR - Self-OPD is a teacher-free on-policy distillation framework that uses a flow matching model’s own stochastic exploration to generate step-wise supervision. It aims to reduce teacher-training costs and distribution mismatch while improving single- and multi-objective alignment.
- At each timestep, the method creates multiple stochastic SDE branches, rolls them out with an ODE sampler, and evaluates their rewards against a deterministic self-reference.
- A pull-push objective attracts the model toward high-advantage branches and repels it from low-advantage ones, with direction-aware attenuation and variance normalization.
- Multi-objective alignment combines normalized scores at the reward level to avoid direct gradient conflicts.
- Reported experiments show improvements over prior reinforcement-learning and OPD methods without task-specific teachers.