🛰️ Daily AI Frontier
‹ back to 2026-07-22

打通感知-理解-交互链路,全栈视频理解大模型VideoChat3开源了

WeChat: 机器之心 Multimodal & Generative 2026-07-22

TL;DR - VideoChat3 is an open-source 4B multimodal model that unifies short-, long-, and streaming-video understanding. Its native spatiotemporal encoding and adaptive visual resolution reduce token and compute costs while supporting timely responses to live video.

  • I3D-ViT jointly models short frame sequences before temporal pooling, achieving roughly 16× spatiotemporal compression.
  • A Silence/Standby/Response mechanism dynamically allocates visual resolution and lets the model decide when sufficient evidence exists to answer.
  • The release includes model weights, code, and three datasets covering academic, long-form, and online video tasks.
  • At 2,048 frames, reported latency fell from Qwen3-VL’s 44.449 seconds to 20.412 seconds, alongside lower FLOPs and memory use.

view merged work →