ReGenVC: End-to-End Real-Time Generative Video Coding at Ultra-Low Bitrate

TL;DR - ReGenVC is a generative talking-head video codec that delivers real-time decoding at roughly one-tenth the bitrate of traditional codecs. It combines a distilled diffusion transformer with distributed inference optimizations to sustain 24 fps on eight GPUs.

• Encodes 77 frames in about 26 kB using a compressed reference frame, pose keypoints, and metadata.
• Uses a four-step distilled diffusion transformer to reconstruct sharp video at ultra-low bitrates.
• Achieves 24 fps through sequence parallelism, a spatially split VAE, and an overlapped three-stage pipeline.
• CPU offloading reduces peak per-GPU memory from 21.1 GB to about 7.7 GB.