FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving
Ranking
Overall
84
Content
90
Popularity
70
Observed public metrics from 1 member.
Merged summary
TL;DR - FlashPrefill V2 advances block-sparse prefill attention toward production-ready long-context LLM serving, combining improved approximation accuracy with optimized kernels and modern serving features. On NVIDIA H20 GPUs at 128K context, it reports speedups of up to 47.26× in FP8 and 27.19× in BF16 over FlashAttention-2.
- A mean correction term limits approximation error, keeping degradation manageable even under extreme sparsity.
- The sparse attention operator uses PackGQA memory access, warp specialization, and ping-pong pipelining aligned with FlashAttention-3/4 designs.
- It supports FP8 inference, paged KV caches, and continuous batching, enabling integration into frameworks such as SGLang.
- In FP8, it achieves a reported 30.49× speedup over an FA3/4-aligned dense baseline at 128K context.
Sources (1)
FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving
Public signals
Hugging Face upvotes 20
TL;DR - FlashPrefill V2 advances block-sparse prefill attention toward production-ready long-context LLM serving, combining improved approximation accuracy with optimized kernels and modern serving features. On NVIDIA H20 GPUs at 128K context, it reports speedups of up to 47.26× in FP8 and 27.19× in BF16 over FlashAttention-2.
- A mean correction term limits approximation error, keeping degradation manageable even under extreme sparsity.
- The sparse attention operator uses PackGQA memory access, warp specialization, and ping-pong pipelining aligned with FlashAttention-3/4 designs.
- It supports FP8 inference, paged KV caches, and continuous batching, enabling integration into frameworks such as SGLang.
- In FP8, it achieves a reported 30.49× speedup over an FA3/4-aligned dense baseline at 128K context.