Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Ranking
Overall
86
Content
95
Popularity
67
Observed public metrics from 1 member.
Merged summary
TL;DR - Minima quantizes every linear layer of a 27B hybrid LLM—including its recurrent Gated DeltaNet layers—to NVFP4 W4A4 while retaining near-BF16 quality. The results challenge the assumption that recurrent layers require higher precision and provide a practical recipe for smaller, faster inference.
- Minima occupies 17.5 GiB, improves prefill speed by 14–19%, and scores within seed noise of BF16 across language, reasoning, coding, and long-context evaluations.
- NVFP4’s 16-element block scaling localizes activation outliers, while nonlinear gate parameterizations reduce roughly 11% projection error to about 2% output error.
- GDN’s delta-rule recurrence prevents quantization noise from compounding: noise plateaus over 32K tokens, and injected state errors are forgotten within hundreds of steps.
- The work fixes scale mismatches in fused serving kernels and recommends quantizing all linear layers while shipping calibrated FP8 KV-cache scales.
Sources (1)
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Public signals
Hugging Face upvotes 83 · Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Minima quantizes every linear layer of a 27B hybrid LLM—including its recurrent Gated DeltaNet layers—to NVFP4 W4A4 while retaining near-BF16 quality. The results challenge the assumption that recurrent layers require higher precision and provide a practical recipe for smaller, faster inference.
- Minima occupies 17.5 GiB, improves prefill speed by 14–19%, and scores within seed noise of BF16 across language, reasoning, coding, and long-context evaluations.
- NVFP4’s 16-element block scaling localizes activation outliers, while nonlinear gate parameterizations reduce roughly 11% projection error to about 2% output error.
- GDN’s delta-rule recurrence prevents quantization noise from compounding: noise plateaus over 32K tokens, and injected state errors are forgotten within hundreds of steps.
- The work fixes scale mismatches in fused serving kernels and recommends quantizing all linear layers while shipping calibrated FP8 KV-cache scales.