FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers

TL;DR - FlashDecoder is a pure-Transformer video decoder that streams latent representations to pixels frame-by-frame, enabling real-time video generation with constant latency and bounded memory regardless of video length.

• Replaces slow, memory-heavy 3D convolutional decoders in latent video diffusion; each frame attends to a fixed-size window of past frames via a rolling KV cache for constant-latency streaming.
• Sequential frame processing enforces temporal causality without explicit attention masks, supporting training at resolutions up to 1080p.
• Matches convolutional decoder reconstruction quality on Wan2.1/Wan2.2 latent spaces (e.g., 41.55dB vs. 41.49dB PSNR at 1080p).
• Decodes 3.6x–4.7x faster with up to 11x less memory on a single H100; architecture-aware inference optimizations widen the speedup to 12x.