Hierarchical Denoising For Multi-Step Visual Reasoning
Merged summary
TL;DR - HDR (Hierarchical Denoising for Visual Reasoning) is a video-generation framework that adds tree-structured, coarse-to-fine latents to causal diffusion so video models can perform multi-step visual reasoning while streaming at low latency. It matters because it narrows the gap between efficient streaming generation and the global, logically consistent reasoning that dense bidirectional diffusion offers but at far lower cost.
- Organizes video latents into a hierarchy: coarse layers hold uncertain hypotheses for global planning, finer layers refine them into concrete visual states; a Sparse Hierarchical Attention Pattern (SHAP) cuts temporal attention cost.
- On a new level-stratified benchmark (maze, Tower of Hanoi, one-line drawing, sliding puzzle, Sokoban, water pouring, with OOD cases), success rises from 34.22 to 60.29 (76.2% relative gain) and average progress from 76.00 to 89.56 vs. streaming autoregressive baselines.
- Maintains 0.70 s/latent streaming (54.2× faster than bidirectional diffusion) and retains 82.9% of full-data performance using only 2% of training data (vs. 52.0% for bidirectional diffusion).
- Reports real-world robot experiments suggesting applicability to physical interaction and world modeling; figures reflect authors' self-reported results.
Sources (1)
Hierarchical Denoising For Multi-Step Visual Reasoning
TL;DR - HDR (Hierarchical Denoising for Visual Reasoning) is a video-generation framework that adds tree-structured, coarse-to-fine latents to causal diffusion so video models can perform multi-step visual reasoning while streaming at low latency. It matters because it narrows the gap between efficient streaming generation and the global, logically consistent reasoning that dense bidirectional diffusion offers but at far lower cost.
- Organizes video latents into a hierarchy: coarse layers hold uncertain hypotheses for global planning, finer layers refine them into concrete visual states; a Sparse Hierarchical Attention Pattern (SHAP) cuts temporal attention cost.
- On a new level-stratified benchmark (maze, Tower of Hanoi, one-line drawing, sliding puzzle, Sokoban, water pouring, with OOD cases), success rises from 34.22 to 60.29 (76.2% relative gain) and average progress from 76.00 to 89.56 vs. streaming autoregressive baselines.
- Maintains 0.70 s/latent streaming (54.2× faster than bidirectional diffusion) and retains 82.9% of full-data performance using only 2% of training data (vs. 52.0% for bidirectional diffusion).
- Reports real-world robot experiments suggesting applicability to physical interaction and world modeling; figures reflect authors' self-reported results.