ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models
作者: Jiahui Ning, Liang Chen, Yuchen Liang, Yi Zhang, Fenglei Fan
分类: cs.AI, cs.CL
发布日期: 2026-10-20
备注: 33 pages, 7 figures, including references and appendices. Code: https://github.com/gaffeyn03/ACE-Compression
💡 一句话要点
提出ACE-Compression框架以评估压缩大语言模型的能力保留
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 模型压缩 能力评估 量化 剪枝 大语言模型 知识分布 能力中心评估
📋 核心要点
- 现有模型压缩评估方法仅关注任务性能，未能揭示压缩对模型能力的深层影响。
- 提出ACE-Compression框架，聚焦于压缩模型的核心能力保留，涵盖多维度能力评估。
- 实验结果显示，量化方法在能力保留上优于剪枝，且能力特征的选择对应用效果至关重要。
📝 摘要（中文）
模型压缩使得大型语言模型可部署，但现有评估方法未能全面反映压缩对模型能力的影响。传统方法仅关注任务性能，忽视了压缩对模型知识分布的扰动。为此，本文提出ACE-Compression，一个以能力为中心的评估框架，旨在探讨压缩模型的核心能力保留情况。通过对六个开源大语言模型的35个压缩检查点进行分析，发现压缩过程重新分配了能力特征，粗糙和表层信号往往得以保留，而细粒度和上下文依赖的能力则变得脆弱。量化方法在能力保留上优于剪枝，但具体效果取决于应用所需的能力特征。
🔬 方法详解
问题定义：本文旨在解决现有模型压缩评估方法的不足，特别是未能揭示压缩对模型能力的深层影响。现有方法仅关注任务性能，忽视了知识分布的变化。
核心思路：ACE-Compression框架通过评估压缩后模型的核心能力，提供更全面的能力保留分析。该框架不仅关注模型是否能正确回答问题，还探讨哪些能力得以保留。
技术框架：该框架包括四个主要维度的能力评估：程序推理与约束执行、知识基础与真实性、社会认知与情感理解、个性忠实度与风格一致性。通过对压缩模型的多维度分析，揭示能力特征的变化。
关键创新：ACE-Compression的最大创新在于其能力中心的评估方法，区别于传统的单一任务性能评估，提供了对模型能力的深入理解。
关键设计：在实验中，使用了超过35个压缩检查点，涵盖量化与剪枝方法，分析了不同压缩策略对能力保留的影响，特别关注能力特征的细粒度变化。通过这些设计，研究者能够更好地理解压缩对模型的影响。
🖼️ 关键图片
📊 实验亮点
实验结果表明，量化方法在能力保留上优于剪枝，尤其在程序推理和知识基础能力方面表现突出。具体而言，量化方法在多个能力维度上保持了较高的性能，提供了更为稳定的能力保留特征。
🎯 应用场景
ACE-Compression框架可广泛应用于大语言模型的部署与优化，帮助开发者根据具体应用需求选择合适的压缩方法。通过理解模型能力的保留情况，开发者能够更有效地设计和调整模型，以满足特定任务的要求，提升实际应用的效果。
📄 摘要（原文）
Model compression makes large language models deployable, but today's evaluation tells an incomplete story. Standard practice measures retained task performance and stops there. This misses a critical fact: compression perturbs the distributed knowledge embedded across billions of weights, and task scores alone cannot reveal which underlying abilities survive and which quietly collapse. To solve this question, we introduce ACE-Compression, an ability-centric evaluation framework. Instead of asking only "does the compressed model still answer correctly?", we ask "which of its core abilities remain intact?" Our framework probes four deployment-facing dimensions: procedural reasoning and constraint execution, knowledge grounding and truthfulness, social cognition and affective understanding, and persona fidelity and stylistic consistency. Across more than 35 compressed checkpoints from six open-weight LLMs-covering quantization and pruning-we find that compression redistributes ability profiles. Coarse, local, and surface-level signals often persist, while fine-grained, long-tail, and context-dependent abilities become fragile. Quantization consistently outperforms pruning in retention, but the practical meaning of that gap depends entirely on which ability an application requires. Based on our findings, deployers can choose compression methods by the ability profile their application needs, not just by a leaderboard average. And algorithm designers gain diagnostic signals to build compression methods that preserve what matters. We believe that ACE-Compression turns compression from a black-box accuracy drop into an interpretable intervention-one we can measure, understand, and ultimately improve.