VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards

📄 arXiv: 2610.26469v1 📥 PDF

作者: Zhizhou Wang, Yaxi Lu, Zhong Zhang, Zikun Hu, Xin Cong, Yankai Lin, Fangming Liu

分类: cs.AI, cs.LG

发布日期: 2026-10-19

备注: 26 pages, 7 figures, including references and appendices. Code: https://github.com/Jeffjeno/VeriCUA

💡 一句话要点

提出VeriCUA以解决GUI代理奖励验证的可靠性问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 主动奖励代理 GUI验证 强化学习 验证策略 计算机视觉

📋 核心要点

  1. 现有方法依赖于提示模型进行验证，但在不同环境下的验证可靠性不足，且无法有效提升。
  2. VeriCUA通过学习单次验证尝试的策略，使用可训练的Judge决定检查内容和停止时机，优化验证过程。
  3. 在OSWorld、MobileWorld和WebArena-Verified等基准测试中，VeriCUA的奖励一致性最高，且在任务成功率上有显著提升。

📝 摘要（中文）

随着计算机使用代理的强化学习日益依赖于主动奖励代理，这些代理与执行后的GUI环境互动，以验证记录的屏幕截图无法揭示的内容。然而，现有方法将验证决策留给提示的基础模型，导致验证可靠性不足。本文提出VeriCUA，一个主动奖励代理，通过单次尝试学习验证策略，优化了验证的正确性和效率。实验结果表明，VeriCUA在多个基准测试中表现优异，显著提高了任务成功率。

🔬 方法详解

问题定义：本文旨在解决现有主动奖励代理在GUI环境中验证可靠性不足的问题。现有方法依赖于提示模型进行验证，但这些模型在不同环境下的表现不稳定，且无法有效提升验证的准确性。

核心思路：VeriCUA的核心思路是通过学习单次验证尝试的策略，使用一个可训练的Judge来决定检查内容和停止时机，从而优化验证过程。这样的设计使得代理能够在执行过程中动态调整验证策略，提高验证的有效性。

技术框架：VeriCUA的整体架构包括两个主要模块：可训练的Judge和冻结的Probe。Judge负责决策，而Probe则在实时环境中执行检查。通过在线强化学习，Judge的决策过程被优化，以提高判决的正确性和Probe的使用效率。

关键创新：VeriCUA的最大创新在于其验证策略的学习方式，而非单纯依赖模型规模。通过优化验证策略，VeriCUA在多个基准测试中超越了现有的提示模型，显示出验证策略在可靠性上的重要性。

关键设计：在设计中，使用了归一化的回报机制来平衡不同终态的奖励，同时在训练过程中确保Judge的决策能够有效指导Probe的执行。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在实验中，VeriCUA在OSWorld、MobileWorld和WebArena-Verified基准测试中实现了最高的奖励一致性，相较于少量示例的GPT-5.4提升了5.36个百分点，同时减少了47.6%-59.8%的Probe令牌使用。此外，VeriCUA在任务成功率上整体提升了3.21点，在多应用任务上提升了5.40点，显示出其显著的性能优势。

🎯 应用场景

VeriCUA的研究成果在多个领域具有广泛的应用潜力，尤其是在自动化测试、用户界面验证和智能代理系统中。通过提高GUI代理的奖励验证可靠性，VeriCUA能够帮助开发者更高效地构建和维护复杂的用户界面应用，提升用户体验和系统稳定性。

📄 摘要（原文）

Reinforcement learning for computer-use agents increasingly relies on proactive reward agents that interact with the post-execution GUI environment to verify what recorded screenshots cannot reveal. The reliability of such verification hinges on two coupled decisions: what to check and when to stop. Existing methods leave these decisions to prompted foundation models. However, neither scaling up the model nor providing few-shot examples reliably improves verification: prompted Judges ranging from 4B to GPT-5.4 all plateau at 68%-70% macro-average agreement. This is because the value of a check depends on the environment rather than the task alone, and becomes apparent only through execution. Learning from execution is itself constrained, as the Actor's terminal state cannot be faithfully replicated during training, allowing only one verification attempt per state. We introduce VeriCUA, a proactive reward agent that learns its verification strategy from these single attempts. A trainable Judge decides what to check and when to stop, while a frozen Probe executes the checks in the live environment. Online reinforcement learning optimizes the Judge for both verdict correctness and efficient Probe use, with returns normalized across terminal states. Trained only on OSWorld and evaluated without further training on MobileWorld and WebArena-Verified, VeriCUA achieves the highest reward agreement across all three benchmarks and lies on the agreement-cost Pareto frontier among proactive verifiers. It surpasses few-shot GPT-5.4 by 5.36 points with 47.6%-59.8% fewer Probe tokens, and even a 1B VeriCUA exceeds every prompted Judge. As the reward for Actor RL, it raises task success by 3.21 points overall and 5.40 points on held-out multi-application tasks. These results identify the verification strategy, not verifier scale, as the key to reliable GUI rewards.