Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models

📄 arXiv: 2609.16589v1 📥 PDF

作者: Keqing Zhang, Jingyu Chen, Yufan Liu, Yongqiang Zhu, Nai Ding, Lai Jiang, Congyan Lang, Bing Li, Weiming Hu

分类: cs.AI

发布日期: 2026-09-15

备注: Preprint. 9 authors

💡 一句话要点

提出PEC框架以解决大语言模型价值观对齐问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大语言模型 价值观对齐 PEC框架 行为分析 人工智能伦理

📋 核心要点

  1. 当前大语言模型在对齐人类价值观方面面临行为悖论，表现出不可预测的波动和对纠正偏见的顽固性。
  2. 本文提出Prior-Environment-Cognition（PEC）框架，通过数学定义价值表达，量化LLMs的内在价值观。
  3. 实验结果表明，PEC框架能够有效验证LLMs的价值观，量化其变化，并实现比传统训练方法更高效的对齐。

📝 摘要（中文）

随着大语言模型（LLMs）处理复杂主观任务的能力增强，将其意图和行为与人类价值观对齐成为一项重要的科学挑战。当前的努力面临行为悖论：在微小措辞变化下表现出不可预测的波动（“摆动”），而对纠正固有偏见的明确指令却表现出顽固的忽视（“刚性”）。为了解决这一双重性，本文通过对106个LLMs（每个模型150,000个查询）和95,000个人类调查档案的实证分析，确认LLMs确实存在内在价值体系，但与人类多样性不符，形成高度集中、理想化的价值核心。我们提出了Prior-Environment-Cognition（PEC）框架，量化这些价值观，并通过适应性“对齐处方”实现价值观的有效对齐。

🔬 方法详解

问题定义：本文旨在解决大语言模型的内在价值观与人类价值观对齐的问题。现有方法在处理模型行为的不可预测性和固有偏见时存在显著不足。

核心思路：提出PEC框架，通过将价值观的表达视为内在倾向、外部环境和内部推理过程的联合结果，系统性地量化和对齐LLMs的价值观。

技术框架：PEC框架包括三个主要模块：Prior（内在倾向）、Environment（外部环境）和Cognition（内部推理），通过这些模块的交互作用来定义和量化价值观。

关键创新：最重要的创新在于PEC框架的提出，它提供了一种新的视角来理解和量化LLMs的价值观，与现有方法相比，能够更有效地识别和调整模型的价值观。

关键设计：在PEC框架中，参数设置包括对模型参数的权重调整、用户提示的设计，以及链式思维过程的优化，以确保对价值观的有效量化和对齐。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，PEC框架能够成功验证LLMs的价值观，并准确量化其变化。与传统的盲目训练方法相比，该方法在对齐效率和精确度上有显著提升，且未降低模型的通用能力。

🎯 应用场景

该研究的潜在应用领域包括人机交互、自动化内容生成和道德决策支持等。通过更好地对齐大语言模型的价值观，可以提高AI系统在复杂任务中的可靠性和安全性，促进其在社会中的广泛应用。

📄 摘要（原文）

As Large Language Models (LLMs) increasingly handle complex subjective tasks, aligning their intentions and behaviors with human values has become a critical scientific challenge. However, current efforts are confounded by a striking behavioral paradox: they fluctuate unpredictably under minor wording changes ("swing"), yet stubbornly ignore explicit instructions to correct ingrained biases ("rigidity"). Resolving this duality is critical for reliable AI alignment. To systematically understand and safely steer these latent subjective preferences, our study is structured around three fundamental questions. First, do LLMs possess an intrinsic value system? By projecting responses from 106 LLMs (150,000 queries per model) and 95,000 human survey profiles into a shared sociological space, we empirically confirm that they do. However, they do not mirror human diversity, instead crystallizing into a highly concentrated, idealized value core. Second, how can these values be quantified? We propose the Prior-Environment-Cognition (PEC) framework. This model mathematically defines value expression as the joint outcome of inherent dispositions like parameter weights (Prior), external contexts such as user prompts (Environment), and internal reasoning processes like Chain-of-Thought (Cognition). Finally, how can LLMs' values be aligned toward a desired target? Using PEC diagnostics, we establish an adaptive "Alignment Prescription". Rather than blindly applying resource-intensive training, this method identifies the minimum effective intervention needed for each dimension, ranging from zero-cost prompts to targeted parameter updates. Extensive empirical validation confirms that our approach successfully verifies the presence of LLM values, accurately quantifies their shifts, and achieves more efficient and precise steering than conventional blind training, all without degrading general capabilities.