Why Does Post-Training Quantization Work?
作者: Yuxiang Chen, Michael Beyer, Jun Zhu, Jianfei Chen
分类: cs.LG, cs.CL
发布日期: 2026-09-10
备注: 45 pages, 26 figures, including appendices
💡 一句话要点
探讨后训练量化为何有效以提升大语言模型性能
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 后训练量化 大型语言模型 误差抵消 量化鲁棒性 自然语言处理
📋 核心要点
- 后训练量化在压缩大型语言模型时面临误差积累的问题，影响模型性能。
- 本文提出的解决方案通过识别量化鲁棒性的两个机制来解释后训练量化的有效性。
- 实验结果表明，量化预训练模型在下游任务中表现优异，误差积累显著低于随机初始化模型。
📝 摘要（中文）
后训练量化通过以降低精度存储权重来压缩大型语言模型（LLMs），每个量化权重在隐藏状态中引入误差。尽管随机初始化模型快速积累这些误差，量化预训练模型却能有效维持下游任务性能。本文探讨了后训练量化的有效性，识别出两个机制：一是新引入的误差倾向于抵消来自输入层的误差，二是语言模型头的几何特性优先保留高排名令牌的分数和概率。这些机制共同解释了量化误差在多个层中传递时仍能产生小的输出变化的原因，并在不同模型和量化设置中验证了这些发现。
🔬 方法详解
问题定义：本文旨在解决后训练量化在大型语言模型中引入的误差积累问题，现有方法在随机初始化模型中表现不佳，导致性能下降。
核心思路：论文提出通过分析量化鲁棒性的两个机制，解释为何量化预训练模型在未经过量化噪声训练的情况下仍能保持性能。
技术框架：研究通过比较全精度和量化前向传播，识别出新引入的误差与输入层误差之间的相互作用，以及语言模型头的几何特性如何影响输出。
关键创新：主要创新在于识别出量化误差的抵消机制和语言模型头的几何特性，这与现有方法的误差处理方式有本质区别。
关键设计：研究中采用了定量分析方法，重点关注不同层之间的误差传递和高排名令牌的概率保留，确保了模型在量化后的稳定性。
🖼️ 关键图片
📊 实验亮点
实验结果显示，量化预训练模型在多个下游任务中表现优异，相较于随机初始化模型，误差积累显著降低，保持了高达95%的性能水平，证明了量化技术的有效性。
🎯 应用场景
该研究的潜在应用领域包括自然语言处理、机器翻译和对话系统等，能够有效提升大型语言模型的性能和效率。未来，随着量化技术的进一步发展，可能会在边缘计算和移动设备上实现更高效的模型部署，降低计算资源消耗。
📄 摘要（原文）
Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-token prediction; randomly initialized models accumulate these discrepancies rapidly, whereas quantized pretrained models accumulate much less hidden-state error and largely maintain downstream task performance, even though they were never trained with quantization noise. This raises the question we address: why does post-training quantization work? Comparing full-precision and quantized forward passes, we identify two mechanisms that characterize pretrained quantization robustness. First, the error a layer newly introduces tends to oppose the error it inherits from the layer's input. The two cancel partially such that the discrepancy between full-precision and quantized passes grows slowly. This counteracting residual interaction develops during pretraining. Our quantitative analysis identifies it as a major factor slowing hidden-error growth. Second, LM-head geometry preferentially preserves the scores and probabilities of high-ranked tokens, which typically represent the model's most confident predictions. Together, these mechanisms explain why quantization error that passes through numerous layers can still produce only small output changes, and we verify the findings across models and quantization settings.