JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied Agents

📄 arXiv: 2610.12957v1 📥 PDF

作者: Gangyi Li, LuYao Wang, Jingfu Sun, Jingyi Yuan, Hang Liu, Hao Zong, Yujia Hu, Tingting Gao, Jianyong Wang, Xu Yang

分类: cs.RO, cs.AI, cs.LG

发布日期: 2026-10-09

备注: 23 pages, 6 figures, 5 tables; includes references and appendices

💡 一句话要点

提出JoyInside-VLM-Agent以解决多模态执行与产品约束问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 具身智能体 多模态学习 产品约束 强化学习 轨迹生成 自蒸馏 执行反馈 机器人技术

📋 核心要点

  1. 现有的具身智能体在处理多模态观察和用户请求时，难以有效适应产品约束和执行反馈，导致执行效率低下。
  2. 本文提出的JoyInside-VLM-Agent通过产品条件轨迹构建和分阶段策略学习，增强了智能体的执行能力和适应性。
  3. 实验结果显示，JoyInside-VLM-Agent在BFCL v4评分中达到77.44，超越了多个基线，并在特定领域的技能精炼上表现优异。

📝 摘要（中文）

具身智能体必须将多模态观察和用户请求转化为可执行动作，同时适应产品约束、变化目标和执行反馈。本文提出了JoyInside-VLM-Agent，这是一种通过产品条件轨迹构建和分阶段策略学习开发的ReAct风格多模态智能体，适用于机器人平台和智能硬件。该系统通过闭环沙盒生成涵盖目标修订和失败恢复的多模态交互轨迹，并根据执行结果和语义标准进行检查。经过监督微调和多教师在线蒸馏后，采用DPER（双源特权在线自蒸馏与令牌级证据路由）进行强化学习。JoyInside-VLM-Agent在BFCL v4整体评分中达到77.44，超越了评估基线，并在JoyInside-Devices和JoyInside-Robotics上在所有报告指标中领先基线。

🔬 方法详解

问题定义：本文旨在解决具身智能体在多模态观察和用户请求转化为可执行动作时的适应性不足问题，现有方法在产品约束和执行反馈下表现不佳。

核心思路：JoyInside-VLM-Agent通过产品条件轨迹构建和分阶段策略学习，结合闭环沙盒生成多模态交互轨迹，提升智能体的执行能力和灵活性。

技术框架：整体架构包括闭环沙盒生成多模态轨迹、监督微调、多教师在线蒸馏和DPER强化学习。主要模块涵盖轨迹生成、策略学习和执行反馈处理。

关键创新：DPER技术通过在特权教师和特权学生之间路由令牌级证据，调节奖励导向的令牌优势，显著提升了学习效率和效果。

关键设计：训练奖励结合生成过程评估、多评审排名和任务队列一致性，确保了多模态学习的有效性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

JoyInside-VLM-Agent在BFCL v4整体评分中达到77.44，超越了所有评估基线。在JoyInside-Devices和JoyInside-Robotics的评估中，该系统在所有报告指标上均领先基线，展示了其卓越的性能。

🎯 应用场景

JoyInside-VLM-Agent的研究成果可广泛应用于智能家居、服务机器人和自动化生产等领域，提升具身智能体在复杂环境中的执行能力和适应性，具有重要的实际价值和未来影响。

📄 摘要（原文）

Embodied agents must translate multimodal observations and user requests into executable actions while adapting to product constraints, changing goals, and execution feedback. We present JoyInside-VLM-Agent, a ReAct-style multimodal agent developed through product-conditioned trajectory construction and staged policy learning for robotic platforms and smart hardware. A closed-loop sandbox generates multimodal interaction trajectories covering goal revisions and failure recovery, which are checked against execution outcomes and semantic criteria. Supervised fine-tuning and multi-teacher on-policy distillation precede reinforcement learning with DPER (Dual-Source Privileged On-Policy Self-Distillation with Token-Level Evidence Routing). DPER routes token-level evidence between a frozen privileged teacher and a privileged student sharing the rollout policy's parameters, modulating reward-derived token advantages while preserving their signs. Training rewards combine generative process evaluation, multi-judge rankings, and task-queue consistency. JoyInside-VLM-Agent achieves a BFCL v4 overall score of 77.44, exceeding the evaluated baselines; results on the other public benchmarks are mixed. On JoyInside-Devices and JoyInside-Robotics, the evaluated system, which includes domain-specific skill refinement, leads the baselines on all reported metrics. Two real-world quadruped case studies illustrate visually grounded search, replanning after execution failure, and explicit failure reporting.