Not All Actions Contribute Equally: Learning Precise Progress Estimation from Imprecise Demonstrations

📄 arXiv: 2610.23769v1 📥 PDF

作者: Huanqian Wang, Boshi Tang, Botong Xiao, Jie Zhang, Yang Yue, Danhua Tao, Shiji Song, Gao Huang

分类: cs.RO

发布日期: 2026-10-16

备注: 9 pages, 13 figures

💡 一句话要点

提出PRIME以解决机器人操作中的进度估计问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱二：RL算法与架构 (RL & Architecture)

关键词: 进度估计 机器人操作 强化学习 加权图 视觉一致性 高精度任务 经验聚合

📋 核心要点

  1. 现有进度估计方法过于依赖专家数据，假设所有动作都对任务成功有贡献，导致在实际操作中出现误判。
  2. PRIME通过构建加权图，利用空间距离和视觉一致性来区分有意义的进展与人类错误，从而实现精确的进度估计。
  3. 在多种精确操作任务的实验中，PRIME表现出色，尤其在高精度场景中，政策成功率提升显著，达到基线的5倍。

📝 摘要（中文）

进度估计器在机器人操作中的强化学习中提供密集的奖励信号，但准确评估任务进度具有挑战性。现有方法过于依赖专家数据，且假设演示中的所有动作都对任务成功有贡献，这在高精度操作中常常不成立。为此，本文提出了PRIME，一个能够从不精确演示中学习精确进度估计的进度估计器。PRIME通过构建加权图来连接观察到的状态，利用空间接近性和视觉一致性来聚合经验。实验表明，PRIME在多种任务中表现出色，尤其在高精度场景中，政策成功率比基线提高了多达5倍。

🔬 方法详解

问题定义：本文旨在解决机器人操作中进度估计的准确性问题。现有方法假设所有演示动作均对任务成功有贡献，导致在高精度操作中误判人类错误为进展。

核心思路：PRIME的核心思路是通过构建加权图来聚合不同轨迹的经验，利用空间接近性和视觉一致性来评估任务进度，从而避免对人类错误的误解。

技术框架：PRIME的整体架构包括状态观察、加权图构建、边权编码和最短路径搜索等模块。通过这些模块，PRIME能够有效地评估任务的进展、停滞和倒退。

关键创新：PRIME的主要创新在于其加权图的构建和最短路径搜索方法，能够在不假设单调进展的情况下，准确区分不同类型的进展。

关键设计：在PRIME中，边权重编码了状态间的空间距离作为转移成本，最短路径搜索用于近似成功的测地距离，确保了进度估计的精确性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在多种精确操作任务的实验中，PRIME的政策成功率比现有的SFT基线提高了多达5倍，尤其在高精度场景中表现尤为突出，显示出其在实际应用中的巨大潜力。

🎯 应用场景

PRIME的研究成果在机器人操作、自动化制造和人机协作等领域具有广泛的应用潜力。通过提高进度估计的准确性，PRIME能够显著提升机器人在复杂任务中的表现，推动智能机器人技术的进步与普及。

📄 摘要（原文）

Progress estimators play an essential role in providing dense reward signals for reinforcement learning in robotic manipulation, but accurately assessing task progress can be much challenging. Existing methods heavily rely on expert data and naively assume that all actions within a demonstration contribute to task success. This assumption often breaks down in real-world scenarios, especially in high-precision manipulation, where subtle geometric errors can lead to task failure. Therefore, these methods can erroneously interpret human mistakes as meaningful progress, further corrupting progress learning. We argue that a key limitation is the lack of a global prior regarding task evaluation criteria, which is crucial for distinguishing meaningful progress from human mistakes. To this end, we propose PRIME, a novel progress estimator capable of learning precise progress estimation from imprecise demonstrations by aggregating experience across trajectories. PRIME constructs a weighted graph over observed states, connecting states across trajectories based on spatial proximity and visual consistency. Edge weights encode spatial distances as transition costs, and shortest-path search approximates the geodesic distance to success. Changes in this distance naturally distinguish advancement, stagnation, and setbacks without assuming monotonic progress over time. Extensive experiments in both simulation and real world on precise manipulation tasks show that PRIME is consistently effective across diverse tasks, and robust to data with varying degrees of suboptimality. Notably, the advantage of PRIME gets more pronounced in higher-precision scenarios, yielding up to a (5\times) improvement in policy success rate over the SFT baselines on charger insertion.