Token Radius Attention for Efficient Video Generation

TL;DR - A training-free sparse-attention method, Token Radius Attention (TRA), cuts the quadratic cost of 3D self-attention in Video Diffusion Transformers by allocating per-query compute budgets from attention entropy. It delivers ~1.6–2x inference speedup on production video models without retraining.

• Core observation: retained attention density varies per query but correlates log-linearly with attention entropy, and dominant interactions form query-centered neighborhoods with token-dependent radii — unlike head/block-level methods that share one budget across queries.
• TRA maps query entropy to an analytic token budget, then converts it into a temporally decayed radius, avoiding explicit key ranking/sorting overhead.
• Systems-level optimizations: fused entropy extraction, warm-up reuse across steps, and block-sparse mask construction to keep overhead low.
• Reported results: 9–19% of attention interactions retained and 1.56x–2.05x speedup across seven Wan2.1, Wan2.2, and HunyuanVideo T2V/I2V configurations with competitive quality; code released.