Collaborative Theory of Mind: Learning to Self-Revise by Anticipating User Responses

📄 arXiv: 2610.12872v1 📥 PDF

作者: Fanzhi Zeng, Anubrata Das, Maytal Saar-Tsechansky, Liu Leqi

分类: cs.AI

发布日期: 2026-10-09

💡 一句话要点

提出协作心智理论以解决用户交互中的自我修正问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 用户交互 自我修正 协作心智理论 多轮对话 机器学习 任务优化

📋 核心要点

  1. 现有大型语言模型在多轮用户交互中表现不足，常常未能有效理解用户需求或修正错误。
  2. 本文提出的协作心智理论（CToM）通过训练模型预测用户反应，进而指导模型自我修正以提高交互质量。
  3. CToM在多个任务中表现优异，特别是在数学推理任务中，准确率显著提升，且减少了用户回复次数，提升了交互效率。

📝 摘要（中文）

大型语言模型（LLMs）在单一任务上表现出色，但在与用户进行多轮交互时仍面临挑战，如在理解用户需求前就尝试解决方案，或在收到反馈后未能修正错误假设。本文提出了协作心智理论（CToM），通过显式训练LLMs预测用户反应，并利用这些预测来改善自身行为。CToM在草拟响应后预测用户的反应，并在呈现响应前进行内部修正。实验表明，CToM在数学推理、编码和写作任务上均优于所有基线方法，最终准确率从74.1%提升至79.6%。

🔬 方法详解

问题定义：本文旨在解决大型语言模型在与用户进行多轮交互时的自我修正能力不足的问题。现有方法往往在未充分理解用户需求时就给出响应，导致交互效率低下。

核心思路：CToM的核心思路是通过训练模型预测用户的反应，并利用这些预测来指导模型在响应前进行自我修正，从而提升交互的有效性和准确性。

技术框架：CToM的整体架构包括草拟响应、预测用户反应和内部修正三个主要模块。在训练过程中，用户对草拟响应的回复被用作预测下一轮的目标。

关键创新：CToM的主要创新在于将用户反应预测与任务性能优化相结合，通过这种方式显著提升了模型的协作能力和交互效率。与传统方法相比，CToM更关注用户反馈的实时利用。

关键设计：在训练过程中，使用负对数似然损失函数来最小化用户回复的预测误差，并结合群体相对策略优化（GRPO）来同时优化任务性能和交互效率。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

实验结果显示，CToM在协作数学推理任务中的最终准确率从74.1%提升至79.6%，相比于传统的GRPO方法，表现出显著的性能提升。此外，CToM在编码和写作任务中，用户回复次数分别减少了8%和17%，同时任务性能也得到了提升。

🎯 应用场景

该研究的潜在应用领域包括智能助手、在线教育和客户服务等场景。通过提升模型的自我修正能力，CToM能够更好地理解用户需求，从而提供更为精准和高效的服务，未来可能对人机交互的质量产生深远影响。

📄 摘要（原文）

Large language models (LLMs) demonstrate strong individual task performance, yet still struggle to collaborate with users over extended, multi-turn interactions, for example by attempting solutions before understanding the user's needs or by failing to revise mistaken assumptions after receiving feedback. We propose Collaborative Theory of Mind (CToM), a framework that addresses these challenges by explicitly training LLMs to anticipate user responses and to use those predictions to improve their own behavior. CToM drafts a response, predicts how the user will respond, and uses this prediction to guide internal revision before presenting a response. During training, the user's reply to each draft serves as the target for predicting the next turn, and we train the assistant to minimize the negative log-likelihood of that reply. We combine this prediction objective with group-relative policy optimization (GRPO) to optimize both task performance and interaction efficiency. Across mathematical reasoning, coding, and writing tasks, CToM achieves higher collaborative task performance than all baselines; for example, on a collaborative mathematical reasoning task, it improves final accuracy from 74.1% to 79.6% compared with naive GRPO. Ablations show that predicting user responses also makes collaboration more efficient: under the same reward, adding prediction supervision reduces the number of user replies by 8% on coding and 17% on writing while improving task performance. Behavioral analyses of interaction trajectories show that user-response predictions become more accurate after training, and that more accurate predictions are associated with larger improvements in collaborative task performance through self-revision.