Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model
作者: Xinghang Li, Jun Guo, Qiwei Li, Long Qian, Hang Lai, Yueze Wang, Hongyu Yan, Jiahang Cao, Xi Chen, Jingen Qu, Jiaxi Song, Nan Sun, Hanye Zhao, Futeng Liu, Wanli Peng, Heyun Wang, Yunhong Wang, Caoyu Xia, Jack Zhao, Diyun Xiang, Hangjun Ye, Heng Qu, Huaping Liu, Jason Li
分类: cs.RO, cs.AI
发布日期: 2026-07-13
💡 一句话要点
提出Xiaomi-Robotics-U0以解决机器人多视图一致性问题
🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 多模态生成 具身合成 机器人技术 图像生成 视频生成 深度学习 人工智能
📋 核心要点
- 现有方法在机器人应用中面临多视图一致性和几何一致性等挑战，限制了其性能和适用性。
- 本文提出的Xiaomi-Robotics-U0模型通过联合优化多种生成任务，扩展了基础图像和视频生成的能力。
- 实验结果显示，该模型在具身场景生成和转移任务中表现优异，成功率显著提高，超越了现有基线。
📝 摘要（中文）
近年来，基础图像和视频生成模型在泛化能力和可控性方面表现出色，但在机器人应用场景中，由于多视图一致性、几何一致性和机器人本体约束等要求，直接应用受到限制。为此，本文提出了Xiaomi-Robotics-U0，一个具有380亿参数的多模态自回归模型，旨在实现统一的具身合成。该模型将具身生成视为基础图像和视频生成的扩展，联合优化文本到图像生成、图像编辑、具身场景生成、具身转移和具身视频生成。Xiaomi-Robotics-U0首次支持高质量的多视图场景生成，并引入结构化、可控的具身转移，保持多视图一致性和交互动态。实验结果表明，该模型在单步和序列生成任务上达到最先进水平，超越了GPT-Image-2.0，并在真实世界的操作任务中显著提高了成功率。
🔬 方法详解
问题定义：本文旨在解决现有基础模型在具身场景生成中的局限性，尤其是多视图一致性和几何一致性的问题。现有方法通常依赖有限的机器人数据，导致视觉知识的损失。
核心思路：Xiaomi-Robotics-U0将具身生成视为基础生成模型的扩展，通过联合优化多种生成任务，保持预训练模型的泛化能力，同时适应具身环境。
技术框架：该模型采用多模态自回归架构，主要模块包括文本到图像生成、图像编辑、具身场景生成、具身转移和具身视频生成，形成一个统一的生成框架。
关键创新：Xiaomi-Robotics-U0是首个支持高质量多视图场景生成的模型，并引入了结构化、可控的具身转移，确保多视图一致性和交互动态的保持。
关键设计：模型包含380亿参数，采用特定的损失函数和网络结构以优化生成质量，具体参数设置和网络细节在论文中详细描述。
🖼️ 关键图片
📊 实验亮点
实验结果表明，Xiaomi-Robotics-U0在单步和序列生成任务上达到了最先进的性能，超越了GPT-Image-2.0，并在World Arena的具身视频生成中排名第一。此外，该模型在真实世界操作任务中的成功率从36.9%提高到63.2%，显示出显著的性能提升。
🎯 应用场景
Xiaomi-Robotics-U0的研究成果在多个领域具有潜在应用价值，包括智能机器人、自动驾驶、虚拟现实和增强现实等。通过实现高质量的具身场景生成，该模型能够提升机器人在复杂环境中的交互能力和适应性，推动智能体的自主学习和决策能力的发展。
📄 摘要（原文）
Recent foundation image and video generation models offer strong generalization and controllability, but their direct application to embodied scenarios is limited by requirements for multi-view consistency, geometric coherence, and robot embodiment constraints. Existing methods typically adapt foundation models with limited robot data, often sacrificing visual knowledge acquired during large-scale pre-training. We present Xiaomi-Robotics-U0, a 38-billion-parameter multimodal autoregressive model for unified embodied synthesis. It treats embodied generation as an extension of foundation image and video generation and jointly optimizes text-to-image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation. This unified framework preserves the generalization of the pre-trained world foundation model while adapting it to embodied settings. Xiaomi-Robotics-U0 is the first model to support high-quality multi-view scene generation across multiple robot embodiments and to introduce structured, controllable embodied transfer for fine-grained editing while preserving multi-view consistency and interaction dynamics. It achieves state-of-the-art results on single-step and sequential generation tasks, outperforming GPT-Image-2.0 in human evaluations of embodied scene generation and transfer, ranking first on World Arena for embodied video generation, and improving the out-of-distribution success rate of pi_0.5 from 36.9% to 63.2% on challenging real-world manipulation tasks. These results show that foundation world models can serve both as embodied world models and scalable data engines for embodied intelligence. Code and checkpoints are available at https://robotics.xiaomi.com/xiaomi-robotics-u0.html.