StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation
Merged summary
TL;DR - StreamHOI is a low-latency framework for streaming, long-duration human–object interaction video generation. Its interaction-aware temporal memory design preserves visual consistency while reaching 17.6 FPS with 0.75-second first-chunk latency.
- Profiles transformer blocks to identify differing memory needs for interaction and surrounding regions.
- Uses bias-guided, memory-specialized training to adapt blocks to tailored historical-memory layouts.
- Adds memory-distance scaling to improve access to early interaction states over long sequences.
- Reports strong interaction plausibility, object fidelity, human quality, and efficiency versus long-video and HOI baselines.
Sources (1)
StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation
TL;DR - StreamHOI is a low-latency framework for streaming, long-duration human–object interaction video generation. Its interaction-aware temporal memory design preserves visual consistency while reaching 17.6 FPS with 0.75-second first-chunk latency.
- Profiles transformer blocks to identify differing memory needs for interaction and surrounding regions.
- Uses bias-guided, memory-specialized training to adapt blocks to tailored historical-memory layouts.
- Adds memory-distance scaling to improve access to early interaction states over long sequences.
- Reports strong interaction plausibility, object fidelity, human quality, and efficiency versus long-video and HOI baselines.