Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration
作者: Deepthika S, Pawni Yadav, B. Sankar
分类: cs.AI, cs.HC
发布日期: 2026-10-22
备注: 49 pages; 32 figures; 7 tables
💡 一句话要点
提出人类依赖指数以解决信任校准崩溃问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 大型语言模型 信任校准 人类依赖指数 错误信息吸收 人机交互 教育技术 智能助手
📋 核心要点
- 现有研究主要关注LLM如何迎合用户期望，而本研究反向探讨用户在LLM自信时是否会接受错误信息。
- 通过设计实验，参与者与五个助手互动，助手的事实信息被不同程度地替换为错误信息，旨在评估用户的信任反应。
- 实验结果显示，用户在面对错误信息时，吸收率高达65%，且对错误信息的信任评分显著提高，表明信任校准的崩溃。
📝 摘要（中文）
本研究探讨大型语言模型（LLMs）在用户信任校准中的作用，特别是当LLM自信且错误时，用户是否会向其倾斜。通过实验发现，参与者在与不同助手互动时，错误信息被吸收的比例高达65%，且对错误信息的信心评分达到8.4/10。研究提出了人类依赖指数，以量化人类对LLM的依赖性，强调信任worthiness是人机交互的属性，并建议学习界面应逐条清晰标示可靠性。
🔬 方法详解
问题定义：本研究旨在解决用户在与大型语言模型（LLM）互动时，如何处理模型自信但错误的信息。现有方法未能充分探讨用户对模型信任的动态变化及其影响。
核心思路：研究通过实验设计，探讨用户在面对不同准确性信息时的反应，尤其是当模型表现出高度自信时，用户是否会倾向于接受错误信息。
技术框架：实验采用了在同一模型（GPT-5-mini）上构建的五个助手，分别替换0%至100%的事实信息。参与者通过聊天学习科学主题，并在每个主题后进行测验。
关键创新：提出了人类依赖指数（Human Deference Index），量化用户对LLM的依赖性，强调信任worthiness是人机交互的属性，且信任评估应考虑双向性。
关键设计：实验中，助手的信心评分始终保持高水平，且在每次测验中提供了正确答案、错误信息及干扰项。参与者的信任评分在任务后几乎没有变化，显示出信任校准的崩溃。实验数据表明，错误信息的吸收率与信心评分之间的关系显著改变。
🖼️ 关键图片
📊 实验亮点
实验结果显示，参与者在104次测验中，错误信息的选择率高达60%，而正确信息的选择率仅为11%。信心评分从7.4降至2.9，表明用户对错误信息的吸收程度显著提高，且信任校准几乎崩溃。
🎯 应用场景
该研究的结果对教育技术、智能助手和人机交互设计具有重要意义。通过理解用户如何接受和处理错误信息，可以改进LLM的设计，使其在提供信息时更加透明和可靠，从而提升用户的信任度和学习效果。
📄 摘要（原文）
Research on sycophancy asks whether large language models (LLMs) bend towards what their users want to hear. We ask the reverse question: when an LLM is confident, consistent, and wrong, do users bend towards it? In a within-subjects experiment, 32 participants learned up to five science topics by chatting with five assistants built on the same model (GPT-5-mini). Unknown to participants, each assistant's briefing had 0, 25, 50, 75, or 100% of its facts replaced by plausible falsehoods, and every assistant spoke with full confidence. A surprise quiz after each topic offered the correct fact, the planted falsehood, and two distractors, with a confidence rating for every answer. Across 104 quizzes and 1,040 answers, scores fell from 7.4 to 2.9 out of 10, and the planted option was chosen on 60% of falsified items against 11% of correctly taught ones (odds ratio 12.5). Transcript coding showed that whenever the assistant stated a falsehood, it was adopted about 65% of the time at every falsification level. Absorbed falsehoods were held with high confidence (8.4/10), the modeled confidence advantage of right over wrong answers vanished, and the discrimination of confidence for correctness fell from an AUROC of 0.70 to chance, while post-task trust ratings barely moved. Participants tended to return to the chat more when the assistant was worse, but returning went with more absorption, not less, and 30% of all turns pasted quiz questions back to the source being checked. The assistant itself was almost never sycophantic and never conceded under pushback. We introduce a baseline-corrected Human Deference Index to quantify this human-side sycophancy and argue that trustworthiness is a property of the human-AI pair, that sycophancy evaluations should measure both directions, and that learning interfaces must make reliability legible claim by claim.