WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Ranking
Overall
78
Content
95
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - WIDE dynamically prunes attention-head and FFN-channel groups for each token, improving LLM inference efficiency while retaining more quality than coarse-grained pruning. Its kernel co-design delivers practical prefill and decoding acceleration.
- Supports token-level dynamic width pruning during both prefill and decoding.
- Uses a two-stage differentiable training pipeline to learn token-wise sparse execution.
- At 50% sparsity, reports a 55.1% performance boost over dynamic depth pruning under calibration-only settings.
- Achieves up to 1.98× prefill and 4.95× decoding kernel speedups, with 1.68× and 1.55× end-to-end acceleration.
Sources (1)
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - WIDE dynamically prunes attention-head and FFN-channel groups for each token, improving LLM inference efficiency while retaining more quality than coarse-grained pruning. Its kernel co-design delivers practical prefill and decoding acceleration.
- Supports token-level dynamic width pruning during both prefill and decoding.
- Uses a two-stage differentiable training pipeline to learn token-wise sparse execution.
- At 50% sparsity, reports a 55.1% performance boost over dynamic depth pruning under calibration-only settings.
- Achieves up to 1.98× prefill and 4.95× decoding kernel speedups, with 1.68× and 1.55× end-to-end acceleration.