DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decoding
Ranking
Overall
82
Content
100
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - DeltaLog speeds linear-attention decoding by storing recent recurrent-state updates in a bounded log and periodically merging them into a dense base state. This preserves model semantics while reducing memory traffic and improving serving performance.
- Replaces most full recurrent-state write-backs with compact update appends.
- Supports GDN, KDA, and RWKV6 in a prototype serving stack.
- Accelerates recurrent-state update kernels by up to 1.86× and cuts write traffic by up to 7.83×.
- Delivers 1.05–1.20× end-to-end speedups over dense recurrent baselines.
Sources (1)
DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decoding
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - DeltaLog speeds linear-attention decoding by storing recent recurrent-state updates in a bounded log and periodically merging them into a dense base state. This preserves model semantics while reducing memory traffic and improving serving performance.
- Replaces most full recurrent-state write-backs with compact update appends.
- Supports GDN, KDA, and RWKV6 in a prototype serving stack.
- Accelerates recurrent-state update kernels by up to 1.86× and cuts write traffic by up to 7.83×.
- Delivers 1.05–1.20× end-to-end speedups over dense recurrent baselines.