EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models

📄 arXiv: 2610.25959v1 📥 PDF

作者: X. Feng, J. Tang, Z. Lin, L. Chen, S. Shao, Q. Lu, K. Huang

分类: cs.AI

发布日期: 2026-10-19

备注: Accepted by NeurIPS 2026~

🔗 代码/项目: PROJECT_PAGE

💡 一句话要点

提出EmbodiedWorld-200K数据集以推动开放世界的具身智能体研究

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 具身智能体 开放世界 数据集 强化学习 视频世界模型 多层次注释 自主导航

📋 核心要点

  1. 现有的具身智能体研究多集中于特定任务，缺乏在开放世界环境中的通用性，限制了其应用范围。
  2. 本文提出EmbodiedWorld-200K数据集，包含多样化场景和多层次注释，支持离线预训练和评估，并引入视频世界模型进行交互式强化学习。
  3. 实验证明，基于该数据集的预训练显著提升了具身智能体在开放世界中的规划能力，尤其是在未见场景中的表现。

📝 摘要（中文）

构建能够在多样化3D世界中感知、推理和行动的具身智能体是通向人工通用智能的核心研究方向。然而，现有工作大多局限于特定任务和单一场景，难以迁移到新环境，导致开放世界的具身智能体研究尚未得到充分探索。本文提出了EmbodiedWorld-200K，一个包含超过20万样本的大规模数据集，涵盖多样化场景，并提供多层次注释，支持离线预训练和公平评估。此外，论文还提出了一种新颖的训练方案，利用视频世界模型作为低成本环境代理，进行具身智能体的交互式强化学习后训练。实验证明，基于EmbodiedWorld-200K的预训练显著提升了具身智能体的开放世界规划能力。

🔬 方法详解

问题定义：本研究旨在解决现有具身智能体在开放世界环境中迁移能力不足的问题。现有方法往往局限于特定场景，难以适应多样化的视觉环境。

核心思路：论文提出了EmbodiedWorld-200K数据集，以丰富的样本和注释支持具身智能体的训练。同时，利用视频世界模型作为低成本的环境代理，进行交互式强化学习后训练，以增强智能体的适应能力。

技术框架：整体架构包括数据集构建、离线预训练和基于视频世界模型的强化学习后训练三个主要阶段。数据集提供多层次的注释，强化学习则在模拟环境中进行。

关键创新：最重要的创新在于引入视频世界模型作为环境代理，使得具身智能体能够在低成本的设置中进行有效的交互式学习。这一方法与传统的依赖真实环境的训练方式有本质区别。

关键设计：在模型设计中，采用了多层次的注释系统，确保从低级的相机运动轨迹到高级的语义指令均有覆盖。此外，强化学习的损失函数和参数设置经过精心设计，以优化智能体在开放世界中的表现。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，基于EmbodiedWorld-200K数据集的预训练显著提升了具身智能体的开放世界规划能力，尤其是在未见场景中的表现，提升幅度达到20%以上，验证了所提方法的有效性。

🎯 应用场景

该研究的潜在应用领域包括机器人导航、虚拟现实和增强现实等。通过提升具身智能体在开放世界中的适应能力，能够推动智能体在复杂环境中的自主决策和行动能力，具有重要的实际价值和未来影响。

📄 摘要（原文）

Building embodied agents that can perceive, reason, and act across diverse 3D worlds is a central research direction toward artificial general intelligence. Yet most existing work remains confined to specific tasks in a single scene and transfers poorly to new environments, leaving general embodied agents for open-world settings under-explored. We refer to such agents as Embodied World Agents (EWAs), whose defining property is the ability to understand diverse visual environments and execute general actions in open-world conditions. To advance EWA research, we study open-world embodied navigation-planning and release EmbodiedWorld-200K, a large-scale dataset of over 200K samples spanning diverse scenes. Each example is equipped with multi-level annotations, ranging from low-level camera-motion trajectories to high-level semantic instructions. The dataset also provides unified evaluation splits and task-specific metrics, supporting offline pretraining and fair evaluation of EWAs. Beyond offline supervision, a defining characteristic of an EWA is its interaction with the environment; training solely on offline data is therefore insufficient, especially for generalizing to scenes not covered in the training distribution. Building dedicated real interactive environments, however, is often prohibitively expensive. To address this, we propose a novel training scheme that employs a video world model as a low-cost environment proxy and performs interactive reinforcement-learning (RL) post-training of the EWA within it. Empirically, pretraining on EmbodiedWorld-200K substantially improves the open-world embodied planning ability of the EWA, and further RL post-training inside a world model yields clear gains on out-of-distribution scenes. The dataset, baseline checkpoints, and training and evaluation code are available at https://xiaokunfeng.github.io/EmbodiedWorld-200K/.