BaKron: Efficient Quantization with Kronecker-Factored Hessians
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - BaKron is an efficient solver for GPTQ-style adaptive-rounding quantization that uses two-sided Kronecker-factored Hessian approximations, capturing output-coordinate correlations that one-sided activation-based methods miss — at the same cubic cost as GPTQ. It matters because it makes richer curvature information practical for post-training quantization of large models.
- Builds on the two-sided adaptive-rounding formulation of BoA and YAQA, which is normally prohibitive because applying GPTQ directly in the vectorized weight domain costs $O(m^2n^2)$.
- Combines anti-diagonal parallelism with a recursive divide-and-conquer construction: for an $m\times n$ weight matrix, $O(m+n)$ sequential steps and total work reduced to $O(mn(m+n))$, matching GPTQ's cubic scaling.
- Modular with respect to both the base quantizer and the Hessian estimator, so it can be paired with a range of Hessian approximations.
- Paper reports practical benchmarks, an efficient technique for computing the relevant Hessians, and experimental evaluation; specific accuracy/speed numbers are not given in the abstract.
Sources (1)
BaKron: Efficient Quantization with Kronecker-Factored Hessians
TL;DR - BaKron is an efficient solver for GPTQ-style adaptive-rounding quantization that uses two-sided Kronecker-factored Hessian approximations, capturing output-coordinate correlations that one-sided activation-based methods miss — at the same cubic cost as GPTQ. It matters because it makes richer curvature information practical for post-training quantization of large models.
- Builds on the two-sided adaptive-rounding formulation of BoA and YAQA, which is normally prohibitive because applying GPTQ directly in the vectorized weight domain costs $O(m^2n^2)$.
- Combines anti-diagonal parallelism with a recursive divide-and-conquer construction: for an $m\times n$ weight matrix, $O(m+n)$ sequential steps and total work reduced to $O(mn(m+n))$, matching GPTQ's cubic scaling.
- Modular with respect to both the base quantizer and the Hessian estimator, so it can be paired with a range of Hessian approximations.
- Paper reports practical benchmarks, an efficient technique for computing the relevant Hessians, and experimental evaluation; specific accuracy/speed numbers are not given in the abstract.