LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

TL;DR - LoopSpec is a training-free, self-speculative decoding framework that uses intermediate recurrent states in Looped Transformers to draft tokens while pipelining their generation with verification. It delivers up to 6.83× inference speedup on reasoning and coding benchmarks while preserving exact decoding behavior.

• Generates draft predictions from early recurrent depths without requiring a separate draft model.
• Overlaps future-token drafting with current-token verification to reduce decoding latency.
• Selectively adds a second proposal from a deeper recurrent state to improve draft accuracy with limited compute overhead.
• Supports lossless greedy and sampling-based decoding, with proposal depths derived in closed form and validated empirically.