Oilbird: Training-Free Speculative Decoding with Keys the Verifier Already Computes
Ranking
Overall
76
Content
100
Popularity
19
Observed public metrics from 1 member.
Merged summary
TL;DR - Oilbird is a training-free speculative decoding method that uses verifier-computed hidden states to retrieve semantically matching drafts missed by exact suffix lookup. It improves accepted draft length and reaches 4.4× autoregressive decoding speed on API-Bank.
- Re-keys an existing context pool with hidden states already computed at each committed token.
- Targets tool-calling workloads where repeated requests differ mainly in newly generated values.
- Improves accepted length by 24–29% across three published drafters at matched pool size and budget.
- Achieves 4.4× speedup on API-Bank, versus 3.9× for the strongest training-free baseline and 2.0× for EAGLE-3.
Sources (1)
Oilbird: Training-Free Speculative Decoding with Keys the Verifier Already Computes
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Oilbird is a training-free speculative decoding method that uses verifier-computed hidden states to retrieve semantically matching drafts missed by exact suffix lookup. It improves accepted draft length and reaches 4.4× autoregressive decoding speed on API-Bank.
- Re-keys an existing context pool with hidden states already computed at each committed token.
- Targets tool-calling workloads where repeated requests differ mainly in newly generated values.
- Improves accepted length by 24–29% across three published drafters at matched pool size and budget.
- Achieves 4.4× speedup on API-Bank, versus 3.9× for the strongest training-free baseline and 2.0× for EAGLE-3.