Structured Transforms for Low-Overhead Quantization of Language Models

TL;DR - Kashin-DCT is a language-model weight quantization method that replaces costly dense random transforms with sign-randomized DCTs, cutting each iteration from $\mathcal{O}(N^2)$ to $\mathcal{O}(N \log N)$. It enables efficient 4-bit-per-channel compression while improving numerical stability on configurations where competing methods fail.

• A greedy alternating-update algorithm produces the four-peak distributions needed to cluster each factor into stable 2-bit codes.
• Closed-form cluster-center initialization eliminates the multi-restart k-means bottleneck used by earlier approaches.
• Combined with OPTQ-style error compensation and QuIP-style incoherence preprocessing, the JAX implementation is competitive on OPT, Llama-2, and Pythia models with favorable wall-clock scaling.
• The bounded-$\ell_\infty$ factorization remains near FP16 perplexity in reported Pythia-6.9B and Mistral-7B stress tests where QuIP variants diverge or encounter NaNs.