DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse

TL;DR - DiTango is a parallel inference framework for Diffusion Transformers that cuts communication overhead in multi-node setups by selectively reusing attention states, delivering faster generation without sacrificing quality.

• Observes that Context Parallelism sequence partitions are heterogeneous: spatially proximate partitions contribute more to attention results, which can be mapped onto hierarchical communication topology to lower cost.
• Introduces a selective attention state mechanism that balances partial attention computation against reusing historical results across denoising steps.
• Uses an anchor-guided state selection planner plus a state-centric runtime to optimize per-partition compute-vs-reuse decisions.
• Reports up to 1.9x end-to-end and 3.2x attention speedup with near-linear multi-node scaling, while maintaining quality comparable to state-of-the-art methods.