MoRoute: Dynamic Routing for In-Context Multimodal Video Generation
Ranking
Overall
68
Content
80
Popularity
40
Observed public metrics from 1 member.
Merged summary
TL;DR - MoRoute is a unified multimodal video generation/editing framework that links a frozen vision-language model to a pretrained video diffusion transformer via dynamic, block-wise layer routing. It matters because it lets heterogeneous pretrained backbones be reused without joint architecture-matched training.
- Treats the frozen VLM and pretrained video DiT as heterogeneous experts; a lightweight block-wise router lets each DiT block pick the most relevant VLM layer for its generation stage.
- Avoids prior limitations: injecting features from only final/manually selected VLM layers, or jointly training architecture-matched understanding and generation streams.
- Uses unified in-context conditioning, feeding reference images and source videos directly into the DiT token sequence to preserve fine-grained visual detail across generation and editing tasks.
- Reported gains over the best competing method on IntelligentVBench, OpenVE-Bench, and RefVIE-Bench: +0.15, +0.18, and +0.34 average score on a 1-5 scale.
Sources (1)
MoRoute: Dynamic Routing for In-Context Multimodal Video Generation
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - MoRoute is a unified multimodal video generation/editing framework that links a frozen vision-language model to a pretrained video diffusion transformer via dynamic, block-wise layer routing. It matters because it lets heterogeneous pretrained backbones be reused without joint architecture-matched training.
- Treats the frozen VLM and pretrained video DiT as heterogeneous experts; a lightweight block-wise router lets each DiT block pick the most relevant VLM layer for its generation stage.
- Avoids prior limitations: injecting features from only final/manually selected VLM layers, or jointly training architecture-matched understanding and generation streams.
- Uses unified in-context conditioning, feeding reference images and source videos directly into the DiT token sequence to preserve fine-grained visual detail across generation and editing tasks.
- Reported gains over the best competing method on IntelligentVBench, OpenVE-Bench, and RefVIE-Bench: +0.15, +0.18, and +0.34 average score on a 1-5 scale.