FlexUMM: Flexible Representation Learning for Unified Multimodal Models

📄 arXiv: 2610.22682v1 📥 PDF

作者: Yinyi Luo, Haoyue Bai, Yi Ge, Nilton Cesar Rojas Vales, Marios Savvides, Jindong Wang

分类: cs.AI

发布日期: 2026-10-15

🔗 代码/项目: GITHUB

💡 一句话要点

提出FlexUMM以解决统一多模态模型的表示冲突问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

📋 核心要点

  1. 现有的统一多模态模型在理解与生成能力之间存在冲突，导致表示不一致和性能下降。
  2. FlexUMM框架通过动态决策共享计算和专用能力的使用，优化了多模态模型的表现。
  3. 实验结果显示，FlexUMM在准确性和计算效率上均有显著提升，减少了功能间的竞争。
  4. method_zh

📝 摘要（中文）

统一多模态模型（UMMs）能够在单一框架内整合理解与生成能力。然而，这两种能力之间的冲突表示常常导致性能下降。现有的UMMs通常依赖于固定结构，未能有效共享计算资源。本文提出FlexUMM，一个灵活的UMM框架，能够动态决定何时保留共享计算、何时路由专用能力以及何时停止进一步处理。实验结果表明，FlexUMM在准确性和效率上均有显著提升，减少了功能间的竞争，并避免了不必要的计算。该框架作为模型无关的功能插件，可以增强现有UMMs的统一性和效率。

🖼️ 关键图片

fig_0
fig_1
fig_2

📄 摘要（原文）

Unified multimodal models (UMMs) can integrate understanding and generation within a single backbone. However, it is commonly known that these two capabilities can cause conflicting representations, leading to a central question: how representation and computation should be shared across functionalities? Existing UMMs often rely on a largely fixed structure, where different inputs and functionalities follow the same depth and parameter-sharing pattern. We argue that this assumption is too restrictive: more effective unification requires sharing the computation that supports cross-functional communication while specializing the capacity that becomes task- or input-specific. We introduce FlexUMM, a flexible UMM framework that dynamically decides when to preserve shared computation, when to route specialized capacity, and when to stop further processing. Evidence shows that this flexible allocation improves UMMs along two complementary axes. It improves accuracy by reducing harmful competition between functionalities, and it improves efficiency by avoiding unnecessary computation once an input representation is sufficiently mature. Beyond simple architectural unification from the pre-trained models, FlexUMM works as a model-agnostic, functional plug-in to make existing UMMs more unified, stronger, and efficient. Code is available at: https://github.com/AIFrontierLab/TorchUMM/tree/main/src/umm/post_training/flexumm.