‹ back to 2026-08-12

StreamFlow: Dynamic Memory Flows for Streaming Video Understanding

arXiv cs.CV Multimodal & Generative Muxin Fu, Yifan Zhang, Wentao Zhang, Fangming Guo, Qian Chen, Guibin Zhang, Shuicheng Yan, Bo An 2026-08-11

TL;DR - StreamFlow is a visual memory framework that lets multimodal LLMs handle streaming video under causal, bounded-memory constraints without modifying the backbone, using dynamic on-demand retrieval of past visual evidence. It matters because it improves both accuracy and efficiency on streaming video understanding, a bottleneck for real-time multimodal agents.

Two-tier memory: a lightweight, dynamics-aware mid-term memory filters temporally redundant frames before visual encoding, while a latent long-term memory consolidates history into visual latents for later reasoning.
An attention-guided retrieval mechanism injects relevant visual latents during generation, triggered when the model's reliance on visual evidence weakens.
Reports 67.73% overall accuracy on StreamingBench (claimed state-of-the-art) plus strong offline long-video results.
Efficiency gains vs. the vanilla setting: +59.1% visual attention score, −50.4% end-to-end latency, −21.1% peak memory.

view merged work →