BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transport

📄 arXiv: 2609.27615v1 📥 PDF

作者: Yanbing Wang, Shenyue Wang, Chunyang Yu

分类: cs.AI, cs.SD

发布日期: 2026-09-23

💡 一句话要点

提出BiCFlow-MER以解决多模态情感识别中的信息融合问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态情感识别 生成模型 条件流 情感空间 音频-文本融合

📋 核心要点

  1. 现有的多模态情感识别方法在处理音频和文本信息时，往往无法有效整合模态特定线索和冲突信息。
  2. BiCFlow-MER提出了一种条件流框架，通过生成证据传输将音频-文本情感识别形式化为结构化情感空间中的过程。
  3. 在IEMOCAP、MELD和零样本CASE基准测试中，BiCFlow-MER的表现超越了所有对比方法，展现了其有效性。

📝 摘要（中文）

在多模态情感识别（MER）中，通过整合来自多种模态的互补线索来推断人类情感状态。然而，音频-文本情感识别中的情感线索常常与说话者风格和词汇内容交织在一起，跨模态的不一致性进一步复杂化了证据的整合。传统的判别融合方法将多模态证据压缩为最终预测，导致模态特定线索和冲突信息的不足保留。为了解决这些问题，本文提出了BiCFlow-MER（双向条件流框架），将音频-文本MER形式化为结构化情感空间中的生成证据传输。该框架通过双向修正流将每个话语传输到明确的情感空间端点，并通过适应性原型云评分和类到条件一致性进行情感验证，显著提升了识别性能。

🔬 方法详解

问题定义：本文旨在解决多模态情感识别中信息融合不足的问题，现有方法无法有效保留模态特定线索和冲突信息，导致情感识别性能受限。

核心思路：BiCFlow-MER通过条件流框架，将音频-文本情感识别视为在结构化情感空间中的生成证据传输，旨在解耦情感证据与说话者风格和词汇内容的影响。

技术框架：BiCFlow-MER的整体架构包括情感导向证据的解耦、双向修正流的传输过程，以及通过适应性原型云评分和类到条件一致性进行情感验证的模块。

关键创新：该框架的核心创新在于通过条件传输将判别识别与生成证据建模相结合，定义了一种新的多模态情感识别范式，显著提升了情感识别的准确性和可靠性。

关键设计：在模型设计中，采用了双向修正流来实现情感空间的传输，并通过适应性原型云评分来验证候选情感，确保了情感识别的冲突感知能力。损失函数和网络结构的设计也针对性地优化了模型性能。

🖼️ 关键图片

fig_0
img_1
img_2

📊 实验亮点

在IEMOCAP、MELD和零样本CASE基准测试中，BiCFlow-MER的性能超越了所有对比方法，显示出显著的提升，尤其在情感识别的准确性和冲突感知能力方面，展现了其创新的有效性。

🎯 应用场景

BiCFlow-MER在多模态情感识别领域具有广泛的应用潜力，尤其是在社交媒体分析、情感计算和人机交互等场景中。通过更准确的情感识别，该技术能够提升用户体验，促进情感理解和沟通。未来，随着技术的进一步发展，BiCFlow-MER可能会在智能客服、心理健康监测等领域发挥重要作用。

📄 摘要（原文）

In multimodal emotion recognition (MER), human affective states are inferred by integrating complementary cues from multiple modalities. In audio-text MER, affective cues are often entangled with speaker style and lexical content, while cross-modal disagreement further complicates how the evidence should be integrated. Under conventional discriminative fusion, multimodal evidence is compressed into a terminal prediction, with modality-specific cues and conflict information insufficiently preserved. In large generative affective models, by contrast, affective reasoning is typically embedded in language decoding, leaving emotion evidence implicit and difficult to verify in a structured space. To address these limitations, BiCFlow-MER (Bidirectional Conditional Flow for Multimodal Emotion Recognition) is proposed as a conditional-flow framework in which audio-text MER is formulated as generative evidence transport within a structured emotion space. Within BiCFlow-MER, emotion-oriented evidence is disentangled from speaker-style and lexical-content factors to construct a conflict-aware affective condition. Guided by this condition, each utterance is transported to an explicit emotion-space endpoint through a bidirectional rectified flow. Candidate emotions are jointly verified through adaptive prototype-cloud scoring of the transported endpoint and backward class-to-condition consistency with the original multimodal condition, enabling conflict-aware recognition. BiCFlow-MER is shown to outperform all compared methods across IEMOCAP, MELD, and the zero-shot CASE benchmark. By orchestrating discriminative recognition and generative evidence modeling through conditional transport, BiCFlow-MER defines a new MER paradigm.