Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

TL;DR - Mask Forcing improves autoregressive video diffusion distillation by perturbing student rollouts with spatial-temporal masks that mix cleaner signals into noisy inputs. This aims to reduce mode collapse, error accumulation, over-saturation, and over-smoothing without real video data or extra post-training.

• Targets reverse-KL mode-seeking in Distribution Matching Distillation, which can collapse the student onto a limited subset of teacher modes.
• Applies random spatial and temporal masks during self-rollout to encourage broader exploration of the teacher distribution.
• Uses cleaner tokens to guide denoising of noisier tokens and improve intermediate rollout predictions.
• Reportedly improves visual quality across multiple autoregressive video diffusion distillation methods while remaining efficient.