‹ back to 2026-07-17

VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

arXiv cs.CV Multimodal & Generative Xinhao Li, Yuhan Zhu, Xiangyu Zeng, Yuhao Dong, Haoning Wu, Zhiqiu Zhang, Yuandong Yang, Changlian Ma, Qingyu Zhang, Yansong Shi, Xinyu Chen, Haoran Chen, Zizheng Huang, Jun Zhang, Kun Ouyang, Lin Sui, Ziang Yan, Yicheng Xu, Chenting Wang, Yinan He, Hongjie Zhang, Yi Wang, Yu Qiao, Yali Wang, Ziwei Liu, Kai Chen, Limin Wang 2026-07-16

TL;DR - VideoChat3 is a fully open-source, 4B-parameter video multimodal LLM designed for efficient and generalist video understanding across general, long-form, and streaming scenarios. It matters because it releases code, training strategy, and datasets to improve reproducibility while claiming to beat equal- or larger-sized open models.

Introduces two efficiency-focused designs: an Inflated 3D Vision Transformer (I3D-ViT) and Adaptive Frame Resolution for Streaming Video Perception, targeting cheaper spatiotemporal representation during training and inference.
Contributes a scalable video data synthesis pipeline producing three datasets—VideoChat3-Academic2M, VideoChat3-LV116K, and VideoChat3-OL617K—spanning general, long-form, and streaming video.
Claims to surpass prior open-source models of equal or larger parameter counts using only 4B parameters with higher efficiency, per benchmarks across general, long-form, and streaming tasks.
Emphasizes being "fully open" (training code, strategy, and datasets), addressing reproducibility gaps in partially open models; specific numeric results are not provided in the content.

view merged work →