UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry

📄 arXiv: 2610.30991v1 📥 PDF

作者: Jialin Tian, Zhuofan Li, Xinkai Wang, Bokai Lin, Chenyi Wang, Xinyu Zhan, Cewu Lu, Lixin Yang

分类: cs.RO

发布日期: 2026-10-22

备注: 14 pages, 5 figures, 4 tables. Project page: https://kksham111.github.io/UniFlowWAM_ProjectPage/

💡 一句话要点

提出UniFlowWAM以解决机器人演示中的3D场景流问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱三：空间感知与语义 (Perception & Semantics)

关键词: 机器人演示 3D场景流 多模态学习 扩散变换器 联合注意力机制 智能机器人 环境理解

📋 核心要点

  1. 现有方法在机器人演示中未能有效整合3D场景流，导致对环境变化的理解不足。
  2. UniFlowWAM提出将3D场景流作为与视觉和动作并列的第三种模态，增强了世界表示的丰富性。
  3. 在RoboCasa-GR1等多个任务中，UniFlowWAM的成功率从52.1%提升至56.4%，显示出显著的性能提升。

📝 摘要（中文）

机器人演示记录了机器人所见和所做的动作，但未明确3D世界在此期间的变化。场景级3D流通过跟踪3D场景点来描述这些变化，提供了具有4D几何的世界表示。现有方法将流作为独立模块或额外预测头，而UniFlowWAM则将3D场景流作为与视觉和动作同等的重要模态。该模型结合了视觉语义和4D几何，并通过联合注意力机制实现模态间的信息交换。实验结果表明，UniFlowWAM在多个任务中显著提高了成功率，展示了其在机器人操作中的有效性。

🔬 方法详解

问题定义：本论文旨在解决现有机器人演示中对3D世界变化的描述不足，现有方法将流作为独立模块，未能有效整合到策略中。

核心思路：UniFlowWAM通过将3D场景流视为与视觉和动作同等的重要模态，构建了一个统一的世界-动作模型，增强了对环境的理解和预测能力。

技术框架：该模型采用扩散变换器架构，每种模态都有一个专家，通过联合注意力机制进行信息交换。模型在十目标角色网格下训练，每个模态被分配为目标、条件或缺失角色。

关键创新：UniFlowWAM的创新在于将3D流纳入策略中，使其成为模型的核心部分，而非附加模块，从而实现了更高效的学习和预测。

关键设计：模型设计中采用了多模态专家机制，并通过特定的损失函数对流进行独立监督，确保各模态间的有效协作。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在RoboCasa-GR1上，UniFlowWAM的成功率从52.1%提升至56.4%，在LIBERO上保持98.1%的成功率，并在LIBERO-Plus上实现87.7%的零-shot性能。在真实机器人实验中，平均成功率达到87.0%，显著优于68.5%的最强基线。

🎯 应用场景

UniFlowWAM在机器人操作、自动化制造和人机交互等领域具有广泛的应用潜力。通过更好地理解和预测环境变化，该模型能够提升机器人在复杂任务中的表现，推动智能机器人技术的进步。

📄 摘要（原文）

Robot demonstrations record what the robot saw and what it did. They do not specify how the 3D world changed in between. Scene-level 3D flow describes it by tracking 3D scene points, providing the world representation with 4D geometry. Prior work has used flow to improve robot policies. Flow served as an intermediate prediction that a separate model converts into actions, as a prior injected at inference time, or as an extra output learned together with the action. In all of these designs, flow stays outside the policy, as a separate module or as an extra prediction head. We introduce UniFlowWAM, a Unified World-Action Model that makes 3D scene flow a third modality on par with vision and action. Its world representation therefore covers both visual semantics, through future visual features, and 4D geometry, through 3D flow, and flow receives its own supervision. UniFlowWAM is a diffusion transformer with one expert per modality, and the three experts exchange information through joint attention. We train UniFlowWAM under a ten-objective role grid, where each objective assigns every modality exactly one of three roles, namely target, condition, or absent. The same model thus serves as a policy, as forward and inverse dynamics, and as a predictor of future visual features or 3D flow. We evaluate UniFlowWAM on RoboCasa-GR1, LIBERO, and LIBERO-Plus in simulation, and on four real-world bimanual manipulation tasks. On RoboCasa-GR1, UniFlowWAM raises the success rate of its base model without flow from 52.1% to 56.4%. It preserves in-distribution success on LIBERO at 98.1% and reaches 87.7% zero-shot on LIBERO-Plus. On the real robot, it achieves 87.0% average success, compared with 68.5% for the strongest baseline.