Unified Static-Dynamic Pruning for Efficient LLM Inference
Merged summary
TL;DR - SPDP combines static weight pruning with input-adaptive dynamic pruning through a shared compressed format and specialized GPU kernels. It accelerates LLM inference while preserving perplexity and supporting greater sparsity.
- Introduces the Tiled-CBC bitmap-compressed format for efficient static and dynamic sparsity.
- Uses a CUDA-core spMspV kernel for activation-aware skipping during decoding.
- Provides a Tensor-Core SpMM kernel optimized for prefill.
- Achieves 1.24–1.37× average speedups, up to 2.51×, over sparse frameworks such as SpInfer, with matching perplexity and up to 25% higher sparsity.
Sources (1)
Unified Static-Dynamic Pruning for Efficient LLM Inference
TL;DR - SPDP combines static weight pruning with input-adaptive dynamic pruning through a shared compressed format and specialized GPU kernels. It accelerates LLM inference while preserving perplexity and supporting greater sparsity.
- Introduces the Tiled-CBC bitmap-compressed format for efficient static and dynamic sparsity.
- Uses a CUDA-core spMspV kernel for activation-aware skipping during decoding.
- Provides a Tensor-Core SpMM kernel optimized for prefill.
- Achieves 1.24–1.37× average speedups, up to 2.51×, over sparse frameworks such as SpInfer, with matching perplexity and up to 25% higher sparsity.