Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Ranking
Overall
80
Content
85
Popularity
67
Observed public metrics from 1 member.
Merged summary
TL;DR - Mask Forcing improves autoregressive video diffusion distillation by perturbing student rollouts with spatial-temporal masks that mix cleaner signals into noisy inputs. This aims to reduce mode collapse, error accumulation, over-saturation, and over-smoothing without real video data or extra post-training.
- Targets reverse-KL mode-seeking in Distribution Matching Distillation, which can collapse the student onto a limited subset of teacher modes.
- Applies random spatial and temporal masks during self-rollout to encourage broader exploration of the teacher distribution.
- Uses cleaner tokens to guide denoising of noisier tokens and improve intermediate rollout predictions.
- Reportedly improves visual quality across multiple autoregressive video diffusion distillation methods while remaining efficient.
Sources (1)
Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Public signals
Hugging Face upvotes 54
TL;DR - Mask Forcing improves autoregressive video diffusion distillation by perturbing student rollouts with spatial-temporal masks that mix cleaner signals into noisy inputs. This aims to reduce mode collapse, error accumulation, over-saturation, and over-smoothing without real video data or extra post-training.
- Targets reverse-KL mode-seeking in Distribution Matching Distillation, which can collapse the student onto a limited subset of teacher modes.
- Applies random spatial and temporal masks during self-rollout to encourage broader exploration of the teacher distribution.
- Uses cleaner tokens to guide denoising of noisier tokens and improve intermediate rollout predictions.
- Reportedly improves visual quality across multiple autoregressive video diffusion distillation methods while remaining efficient.