Self Gradient Forcing: Native Long Video Extrapolation

TL;DR - Self Gradient Forcing (SGF) trains autoregressive video diffusion models to encode earlier generated frames into more useful causal memory. It improves long-video consistency and enables extrapolation from a 5-second training window to several-minute videos.

• Addresses the “historical context-gradient gap,” where future losses cannot improve earlier key-value cache representations.
• Uses a two-pass method: a no-gradient autoregressive rollout followed by parallel context-gradient reconstruction.
• Avoids backpropagation through the full serial rollout while restoring future-to-context supervision.
• Outperforms Self Forcing in identity, layout/background consistency, and temporal stability across long-horizon experiments.