Post-hoc Epistemic Uncertainty for Vision-Language-Action Policies

📄 arXiv: 2610.29730v1 📥 PDF

作者: Johannes Keller, Albrecht M. Bloße, Adrian Lindenmeyer, Thomas Neumuth, Daniel Schneider

分类: cs.RO, cs.LG, stat.ML

发布日期: 2026-10-21

备注: 30 pages, 5 figures, 11 tables. Code: https://github.com/post-hoc-vla-uncertainty/post-hoc-vla-uncertainty

💡 一句话要点

提出后处理内在不确定性估计以优化视觉-语言-行动策略

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 视觉-语言-行动 不确定性估计 主动学习 高斯过程 机器人学习

📋 核心要点

  1. 现有的视觉-语言-行动策略在面对新情况时，难以有效识别知识空白，导致无法及时请求帮助。
  2. 本文提出后处理核方差头，通过单一策略的嵌入估计内在不确定性，避免了多策略训练的复杂性。
  3. 实验表明，核方差头在任务成功率、演示获取和对监督策略的推荐上表现优异，且不确定性评分时间减少了约50倍。

📝 摘要（中文）

视觉-语言-行动（VLA）策略需要识别训练知识中的空白，以便在新情境中请求有用的演示或帮助，尤其是在失败代价高昂的情况下。本文提出的后处理核方差头可以从单一策略的嵌入中估计内在不确定性。每个头计算基于训练演示的嵌入条件下的高斯过程的条件方差，且不需要回归目标或梯度更新。实验结果表明，核方差头在任务成功的任务级相关性、主动学习中的演示获取以及对更强监督策略的推荐方面，与双策略集成得分表现相当，同时显著降低了不确定性评分时间和设置成本。

🔬 方法详解

问题定义：本文旨在解决视觉-语言-行动策略在新情境中识别知识空白的能力不足，现有方法依赖多个独立训练的策略，增加了复杂性和成本。

核心思路：提出后处理核方差头，通过单一策略的嵌入来估计内在不确定性，避免了多策略训练的需求，简化了模型的使用。

技术框架：整体架构包括一个主策略网络和多个核方差头，后者从主策略的嵌入中计算高斯过程的条件方差，形成不确定性估计。

关键创新：最重要的创新在于通过单一策略的嵌入实现内在不确定性估计，显著降低了训练和运行成本，与传统的多策略集成方法相比，提供了更高效的解决方案。

关键设计：核方差头的设计不需要回归目标或梯度更新，保持了主策略的完整性，且在实验中显示出在不确定性评分时间和设置成本上的显著优势。

📊 实验亮点

实验结果显示，核方差头在任务级相关性、演示获取和对监督策略的推荐上表现出色，尤其是在不确定性评分时间上减少了约50倍，相较于传统的双策略集成方法，展现了显著的性能提升。

🎯 应用场景

该研究的潜在应用领域包括机器人学习、自动化系统和人机协作等场景。通过有效估计不确定性，系统能够在面对未知情况时更好地请求帮助，从而提高任务成功率和效率。未来，该方法可能推动智能系统在复杂环境中的自适应能力和决策质量。

📄 摘要（原文）

Vision-language-action (VLA) policies need to recognize gaps in their training knowledge to request useful demonstrations or assistance in new situations, especially when failures are costly. Disagreement within an ensemble of independently trained policies can guide these decisions, but requires training and running multiple policies. We propose post-hoc kernel-variance heads that estimate epistemic uncertainty from the embeddings of a single policy. Each head computes the conditional variance of a Gaussian process conditioned on embeddings from the training demonstrations. Fitting requires no regression targets or gradient updates and leaves the policy unchanged. In experiments with SmolVLA on LIBERO-10, the kernel variance heads perform competitively with two-policy ensemble scores in task-level correlation with task success, in demonstration acquisition during active learning, and in referral to a stronger supervisor policy as a proxy for human assistance. For SmolVLA, the head on the vision-language backbone reduces uncertainty-scoring time by up to approximately a factor of 50 compared with ensemble scores, while also substantially reducing setup costs. We also show that the established evaluation protocol on LIBERO-10 primarily rewards recognition of coverage differences between tasks, while execution variability weakens assessment of within-task rankings. Thus, controlled perturbations provide a complementary evaluation: modality-specific heads identify unfamiliar camera, instruction, or state inputs, while fused heads can detect unfamiliar combinations of individually familiar inputs. With a supervisor proxy trained on the evaluated perturbation families, the decoder head improves within-task referral over ensemble disagreement. These findings show that useful uncertainty estimates for adaptive robot learning can be obtained from a single policy.