Who Are They to Each Other? Multi-Agent Reasoning for Speaker Relationship Inference
作者: Yaohan Guan, Yen-Ju Lu, Yuzhe Wang, Junhyeok Lee, Jesus Villalba, Laureano Moro Velazquez, Thomas Thebaud, Najim Dehak
分类: cs.MA, cs.CL, cs.SD
发布日期: 2026-09-09
💡 一句话要点
提出无训练多智能体推理框架以解决说话者关系推断问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 多智能体推理 说话者关系推断 无训练框架 社交意识理解 多模态交互
📋 核心要点
- 现有方法在推断说话者关系时缺乏有效的结构，且监督学习成本高，难以扩展。
- 论文提出了一种无训练的多智能体推理框架，通过智能体间的互动进行关系判断，避免了特定任务的训练需求。
- 实验结果表明，所提方法在多种模态设置下优于零-shot和现有基线，尤其在文本包含的设置中表现出色。
📝 摘要（中文）
从口语对话中推断说话者关系是实现社会意识语音理解的重要步骤。然而，这一任务仍然未得到充分探索，且监督建模的训练和扩展成本高昂。现有的推理时大语言模型方法在处理微妙、分布式和多模态关系线索方面提供的结构有限，无法支持多种合理解释。为了解决这些局限性，本文提出了一种无训练的多智能体推理框架，通过智能体之间的结构化互动组织推理，允许关系判断的提出、挑战和裁决，而无需特定任务的训练。我们用两种互补设计实例化该框架，提出了多角色多智能体辩论和多智能体竞争，评估结果表明这些方法在大多数情况下优于零-shot和现有的多智能体基线。
🔬 方法详解
问题定义：本文旨在解决从口语对话中推断说话者之间关系的任务。现有方法在处理复杂的关系线索时缺乏有效的结构，且监督学习的训练成本高，难以扩展。
核心思路：论文提出了一种无训练的多智能体推理框架，允许智能体通过结构化互动进行关系判断，避免了特定任务的训练需求。该框架通过多角色和竞争机制增强推理的准确性和可靠性。
技术框架：整体架构包括两个主要模块：多角色多智能体辩论和多智能体竞争。前者为智能体分配互补角色，后者通过成对裁决比较智能体判断，保留最具防御性的候选。
关键创新：最重要的技术创新点在于引入了无训练的多智能体推理机制，允许智能体在推理过程中进行互动和挑战，这与现有方法的单一视角推理方式有本质区别。
关键设计：在多角色多智能体辩论中，智能体被赋予不同的社会理论视角；而在多智能体竞争中，通过成对裁决消除较弱候选，保留最强的判断。
🖼️ 关键图片
📊 实验亮点
实验结果显示，所提方法在Seamless Interaction数据集上，在多种模态设置下均优于零-shot和现有多智能体基线，尤其在文本包含的设置中，LLM方法在某些情况下超越了人类标注者，而在音频设置中表现相对较弱。
🎯 应用场景
该研究的潜在应用领域包括社交机器人、智能客服和语音助手等，能够提升这些系统在理解人际关系和社交动态方面的能力，进而改善用户体验。未来，该框架可能在多模态交互和人机协作中发挥更大作用。
📄 摘要（原文）
Inferring speaker relationships from spoken conversations is an important step towards socially aware speech understanding. However, this task remains underexplored, and supervised modeling is costly to train and scale. At the same time, existing inference-time LLM approaches provide limited structure for handling subtle, distributed, and multimodal relational cues that may support multiple plausible interpretations. To address these limitations, we introduce a training-free multi-agent reasoning framework that organizes inference through structured interaction among LLM agents, allowing relationship judgments to be proposed, challenged, and adjudicated without task-specific training. We instantiate this framework with two complementary designs. We propose Multi-Role Multi-Agent Debate as a task-specific adaptation of standard multi-agent debate for speaker relationship inference, assigning agents complementary roles or social-theory-grounded perspectives rather than a single undifferentiated viewpoint. In contrast, we introduce Multi-Agent Compete, a competition-based protocol that compares agent judgments through pairwise adjudication, eliminates weaker candidates, and retains the most defensible one. We evaluate these methods on the Seamless Interaction dataset across different modality settings, covering both binary classification and fine-grained relationship-detail prediction. Results suggest that they improve over zero-shot and existing multi-agent baselines in most cases. Human evaluation further suggests that this task is challenging even for people. LLM methods can sometimes outperform human annotators in text-included settings but are less competitive in the audio setting. Together, these findings suggest that relationship inference benefits from structured inference-time interaction among agents, while acoustic cues are not yet fully captured by current models.