TreeWY: Speculative Verification for Gated DeltaNet Hybrids
Ranking
Overall
82
Content
100
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - TreeWY is a speculative-verification method for Gated DeltaNet hybrid models that eliminates per-draft-node recurrent-state snapshots. It reduces memory pressure during serving, enabling higher throughput, lower time-to-first-token, or wider draft trees when memory is the bottleneck.
- Uses a tree-structured WY transform and a single triangular solve to compute outputs for all draft nodes.
- Stores a compact pseudo-value matrix and reconstructs only the accepted recurrent state at commit time.
- Benchmarks on Qwen3.5 35B and 397B show reduced recurrent-state memory and KV-cache pressure at unchanged acceptance length.
- Wider, higher-acceptance draft trees become memory-affordable, though they do not yet improve throughput; non-memory-bound cases incur a few percent overhead.
Sources (1)
TreeWY: Speculative Verification for Gated DeltaNet Hybrids
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - TreeWY is a speculative-verification method for Gated DeltaNet hybrid models that eliminates per-draft-node recurrent-state snapshots. It reduces memory pressure during serving, enabling higher throughput, lower time-to-first-token, or wider draft trees when memory is the bottleneck.
- Uses a tree-structured WY transform and a single triangular solve to compute outputs for all draft nodes.
- Stores a compact pseudo-value matrix and reconstructs only the accepted recurrent state at commit time.
- Benchmarks on Qwen3.5 35B and 397B show reduced recurrent-state memory and KV-cache pressure at unchanged acceptance length.
- Wider, higher-acceptance draft trees become memory-affordable, though they do not yet improve throughput; non-memory-bound cases incur a few percent overhead.