Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

📄 arXiv: 2607.12894v1 📥 PDF

作者: Ziyi Wang, Xumin Yu, Yongming Rao, Yonggen Ling, Yunheng Li, Oran Wang, Mingqi Gao, Yuchen Zhou, Yves Liang, Zuyan Liu, Yani Zhang, Rui Huang, Xiaoran Xu, Bowen Yuan, Yifu Yuan, Xu Tan, He Zhang, Yufei Huang, Shenghao Zhang, Hongsheng Wu, Han Hu, Zhengyou Zhang

分类: cs.CV

发布日期: 2026-07-14

备注: Tech Report. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied

💡 一句话要点

提出Hy-Embodied-VLM-1.0以提升物理世界代理的能力

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 具身代理 多模态感知 物理世界理解 推理能力 专家混合架构

📋 核心要点

  1. 现有的具身代理模型在推理和适应动态环境方面存在不足，难以有效处理复杂的物理世界任务。
  2. 本文提出Hy-Embodied-VLM-1.0模型，通过定义行动中心的能力分类法，系统性地培养代理的推理和适应能力。
  3. 在38个基准测试中，该模型在19个基准上超越同类模型，且在多轮交互和长时间推理任务中表现优异。

📝 摘要（中文）

构建高效的具身代理不仅需要多模态感知和理解能力，还需具备推理、适应变化和与物理世界互动的能力。本文介绍了Hy-Embodied-VLM-1.0，这是一个专为物理世界中的具身代理设计的高效强大基础模型。我们定义了一个以行动为中心的能力分类法，涵盖行动相关状态理解、行动转移推理和顺序适应推理三个维度。通过系统的数据管道和数据混合，我们在预训练和后训练阶段培养这些能力。该模型基于Hy3-A3B语言骨干和Hy-ViT2视觉编码器，采用高效的专家混合架构，结合强大的模型能力与高推理效率。在38个基准测试中，Hy-Embodied-VLM-1.0在19个基准上表现最佳，平均性能提升8.4%。

🔬 方法详解

问题定义：本文旨在解决现有具身代理在动态环境中推理和适应能力不足的问题。现有方法在处理复杂物理任务时常常表现不佳，缺乏有效的行动推理机制。

核心思路：Hy-Embodied-VLM-1.0通过定义一个以行动为中心的能力分类法，系统性地培养代理的推理能力，确保其能够在物理世界中有效互动和适应变化。

技术框架：该模型基于Hy3-A3B语言骨干和Hy-ViT2视觉编码器，采用混合专家架构。整体流程包括数据预处理、模型训练和后续评估，涵盖预训练和后训练阶段的数据混合。

关键创新：最重要的创新在于其专家混合架构，能够在激活较少参数的情况下，提供接近于全参数模型的性能，显著提高推理效率。

关键设计：模型激活仅使用3B参数，但通过优化的网络结构和损失函数设计，确保了在多轮交互和长时间推理任务中的高效表现。

📊 实验亮点

在38个基准测试中，Hy-Embodied-VLM-1.0在19个基准上取得最佳表现，较前代模型Hy-Embodied-0.5 MoT-2B平均性能提升8.4%。该模型在多轮交互和长时间推理任务中也展现出强大的能力，显示出其在实际应用中的潜力。

🎯 应用场景

Hy-Embodied-VLM-1.0在机器人、智能家居、自动驾驶等领域具有广泛的应用潜力。其高效的推理和适应能力使得具身代理能够更好地理解和互动物理环境，推动智能系统的实际应用和发展。

📄 摘要（原文）

Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-Embodied-VLM-1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical-world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-Embodied-0.5 MoT-2B, Hy-Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy-Embodied-VLM-1.0 also demonstrates strong performance on embodied agentic tasks requiring multi-turn interaction and long-horizon reasoning.