LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Ranking
Overall
81
Content
90
Popularity
59
Observed public metrics from 1 member.
Merged summary
TL;DR - LoopSpec is a training-free, self-speculative decoding framework that uses intermediate recurrent states in Looped Transformers to draft tokens while pipelining their generation with verification. It delivers up to 6.83Ă— inference speedup on reasoning and coding benchmarks while preserving exact decoding behavior.
- Generates draft predictions from early recurrent depths without requiring a separate draft model.
- Overlaps future-token drafting with current-token verification to reduce decoding latency.
- Selectively adds a second proposal from a deeper recurrent state to improve draft accuracy with limited compute overhead.
- Supports lossless greedy and sampling-based decoding, with proposal depths derived in closed form and validated empirically.
Sources (1)
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Public signals
Hugging Face upvotes 1
TL;DR - LoopSpec is a training-free, self-speculative decoding framework that uses intermediate recurrent states in Looped Transformers to draft tokens while pipelining their generation with verification. It delivers up to 6.83Ă— inference speedup on reasoning and coding benchmarks while preserving exact decoding behavior.
- Generates draft predictions from early recurrent depths without requiring a separate draft model.
- Overlaps future-token drafting with current-token verification to reduce decoding latency.
- Selectively adds a second proposal from a deeper recurrent state to improve draft accuracy with limited compute overhead.
- Supports lossless greedy and sampling-based decoding, with proposal depths derived in closed form and validated empirically.