AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Ranking
Overall
74
Content
90
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - AdaFlash accelerates speculative LLM decoding by stabilizing diffusion-based drafting and dynamically selecting draft length. It reports up to roughly 66% higher throughput than prior state-of-the-art methods, particularly under high concurrency.
- Uses reverse-KL on-policy distillation to reduce domain-level variance and improve training stability.
- Adds an adaptive length head to address position-dependent token quality and reduce target-model verification costs.
- Retains single-pass parallel drafting while mitigating the variability introduced by bidirectional attention.
- Consistently improves deployment speedup across experiments, with the largest gains in high-concurrency settings.
Sources (1)
AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Public signals
Hugging Face upvotes 0 · Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - AdaFlash accelerates speculative LLM decoding by stabilizing diffusion-based drafting and dynamically selecting draft length. It reports up to roughly 66% higher throughput than prior state-of-the-art methods, particularly under high concurrency.
- Uses reverse-KL on-policy distillation to reduce domain-level variance and improve training stability.
- Adds an adaptive length head to address position-dependent token quality and reduce target-model verification costs.
- Retains single-pass parallel drafting while mitigating the variability introduced by bidirectional attention.
- Consistently improves deployment speedup across experiments, with the largest gains in high-concurrency settings.