Why Does Post-Training Quantization Work?

TL;DR - This paper explains why post-training quantization preserves LLM performance despite introducing weight errors at every layer. Pretraining creates error-canceling residual interactions, while LM-head geometry protects the probabilities of highly ranked tokens.

• Newly introduced layer errors tend to oppose inherited errors, slowing hidden-state discrepancy growth.
• This counteracting behavior develops during pretraining and is absent in randomly initialized models.
• LM-head geometry preferentially preserves scores and probabilities for the model’s most confident token predictions.
• The authors verify both mechanisms across multiple models and quantization settings.