StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs

TL;DR - StreamTTT is a streaming vision-language model that stores long-range history in online-updated fast weights while reserving a short attention cache for recent visual evidence. This improves both real-time perception and historical recall without requiring a larger model.

• Separates long-term memory from the attention context to reduce attention dilution.
• Uses a short sliding key-value cache focused on current-scene perception.
• Jointly trains on offline long-video QA and a new real-time QA corpus.
• StreamTTT-4B beats SimpleStream-4B by 1.4 points in real-time perception and 3.7 points in backward tracing on OVO-Bench.