FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving

TL;DR - FlashPrefill V2 advances block-sparse prefill attention toward production-ready long-context LLM serving, combining improved approximation accuracy with optimized kernels and modern serving features. On NVIDIA H20 GPUs at 128K context, it reports speedups of up to 47.26× in FP8 and 27.19× in BF16 over FlashAttention-2.

• A mean correction term limits approximation error, keeping degradation manageable even under extreme sparsity.
• The sparse attention operator uses PackGQA memory access, warp specialization, and ping-pong pipelining aligned with FlashAttention-3/4 designs.
• It supports FP8 inference, paged KV caches, and continuous batching, enabling integration into frameworks such as SGLang.
• In FP8, it achieves a reported 30.49× speedup over an FA3/4-aligned dense baseline at 128K context.