MoRoute: Dynamic Routing for In-Context Multimodal Video Generation

TL;DR - MoRoute is a unified multimodal video generation/editing framework that links a frozen vision-language model to a pretrained video diffusion transformer via dynamic, block-wise layer routing. It matters because it lets heterogeneous pretrained backbones be reused without joint architecture-matched training.

• Treats the frozen VLM and pretrained video DiT as heterogeneous experts; a lightweight block-wise router lets each DiT block pick the most relevant VLM layer for its generation stage.
• Avoids prior limitations: injecting features from only final/manually selected VLM layers, or jointly training architecture-matched understanding and generation streams.
• Uses unified in-context conditioning, feeding reference images and source videos directly into the DiT token sequence to preserve fine-grained visual detail across generation and editing tasks.
• Reported gains over the best competing method on IntelligentVBench, OpenVE-Bench, and RefVIE-Bench: +0.15, +0.18, and +0.34 average score on a 1-5 scale.