Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating

📄 arXiv: 2610.08320v1 📥 PDF

作者: Haozhuo Zhang, Qiang Zhang, Jian Tang, Mingzhe Ni, Michele Caprio, Angelo Cangelosi, Wei Pan

分类: cs.RO, cs.GR

发布日期: 2026-10-06

备注: Videos and results are available at https://haozhuo-zhang.github.io/Humanoid-Horizon-project-page/

💡 一句话要点

提出Humanoid Horizon以解决长时间全身运动操控问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control)

关键词: 类人机器人 全身运动 操控任务 并行训练 动态起始 奖励门控 长时间任务 强化学习

📋 核心要点

  1. 现有方法在长时间全身运动操控任务中存在容易获得奖励的偏差和灾难性遗忘的问题，导致性能下降。
  2. 本文提出的Humanoid Horizon框架通过并行训练、动态起始和奖励门控机制来解决上述问题，确保各阶段均衡训练。
  3. 在LHM-Humanoid基准测试中，成功率超过80%，尽管随着运输物体数量增加，成功率有所下降，但下降幅度相对平缓。

📝 摘要（中文）

在杂乱的室内环境中，类人机器人需要在单一连续的过程中，顺序导航、抓取、运输并准确放置每个物体。这个长时间的全身运动操控任务对现有方法提出了重大挑战。以往的方法常常面临两个主要问题：容易获得奖励的偏差和灾难性遗忘。本文提出Humanoid Horizon，一个统一的策略框架，通过并行训练、动态起始和奖励门控三种机制来克服这些限制。实验结果表明，在两物体的LHM-Humanoid基准测试中，成功率超过80%。

🔬 方法详解

问题定义：本文旨在解决类人机器人在杂乱环境中进行长时间全身运动操控的挑战，现有方法在训练过程中容易导致奖励偏差和灾难性遗忘。

核心思路：Humanoid Horizon框架通过并行训练、动态起始和奖励门控机制，确保机器人在各个阶段均衡学习，避免早期和后期阶段的性能冲突。

技术框架：该框架包括三个主要模块：并行训练策略、动态起始机制和奖励门控。并行训练将多个场景组织成并发阶段流，动态起始则通过更新环境初始状态来增强鲁棒性，奖励门控则控制奖励的分配。

关键创新：最重要的创新在于并行训练策略和奖励门控机制的结合，使得机器人能够在整个任务中保持对已放置物体的稳定性，避免干扰。

关键设计：在并行训练中，$N$个场景被组织为$S$个阶段流，确保所有阶段都能持续接收梯度更新；动态起始机制通过使用上游回放的终态来更新初始状态；奖励门控则在后续阶段中对超出阈值的物体位移设置零奖励。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在LHM-Humanoid基准测试中，Humanoid Horizon框架在两物体任务中成功率超过80%。随着运输物体数量增加，成功率虽然有所下降，但相较于所有基线方法，下降幅度显著更为平缓，显示出该方法的有效性和鲁棒性。

🎯 应用场景

该研究的潜在应用领域包括家庭服务机器人、工业自动化和仓储管理等。通过提升类人机器人在复杂环境中的操控能力，能够有效提高工作效率和安全性，未来可能在智能家居和智能物流等领域发挥重要作用。

📄 摘要（原文）

Cluttered indoor environments, where large and heavy objects are scattered across diverse surfaces, require humanoid robots to sequentially navigate, grasp, transport, and accurately place each item at its target location within a single uninterrupted episode. This long-horizon, whole-body loco-manipulation task remains a significant challenge for current methods. Previous approaches often suffer from two main issues: easy-reward bias, where training overemphasizes early transport stages at the expense of later ones, and catastrophic forgetting, where focusing on later stages leads to a decline in earlier-stage performance. In this work, we introduce Humanoid Horizon, a unified policy framework designed to overcome these limitations through three interrelated mechanisms. The Parallel Training Strategy organizes $N$ scenes into $S$ concurrent stage streams governed by a shared policy, ensuring all transport stages receive continuous gradient updates and removing the bottleneck of sequential optimization. The Dynamic Starting Mechanism updates each environment's initial state with terminal states from upstream rollouts, gradually broadening transition coverage and enhancing robustness at stage boundaries. Reward Gating sets the reward to zero for the rest of the episode in later-stage streams when the immediately preceding object is displaced beyond a set threshold, so the shared policy learns not to disturb a just-placed object and earlier placements are preserved throughout the episode. Collectively, these strategies achieve per-stage success rates exceeding 80\% on the two-object LHM-Humanoid benchmark (350 training scenes, 66 held-out scenes). As the number of sequentially transported objects grows beyond two, success declines with the horizon, but the degradation is graceful relative to the sharp drop seen in all baselines.