Video-HopChain: Multi-Hop Questions and Confidence-Gated Exploration for Video Reasoning Models

📄 arXiv: 2609.25773v1 📥 PDF

作者: Trung Nguyen Quang, Yuhao Dong, Shuo Sun, Shuai Liu, Shulin Tian, Kim-Hui Yap, Ziwei Liu

分类: cs.CV, cs.AI

发布日期: 2026-09-22

💡 一句话要点

提出Video-HopChain以解决视频推理中的多跳问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 视频推理 多跳问题 数据合成 信心门控探索 视觉语言推理 深度学习 模型训练

📋 核心要点

  1. 现有的视觉语言推理方法在视频领域面临多跳推理的挑战，尤其是在长链推理中容易暴露错误。
  2. 本文提出Video-HopChain数据集，通过多跳问题链的设计，增强视频推理模型的学习效果。
  3. 实验结果表明，使用CGE方法后，模型在视频理解和推理基准上的平均分数从57.9提升至59.3，显示出显著的性能改进。

📝 摘要（中文）

HopChain在静态图像上显示了多跳数据合成能改善视觉语言推理，但在视频领域尚未得到探索。为此，本文构建了Video-HopChain数据集，包含22,550个多跳视频问题，涉及13,378个视频。每个问题由三个到六个是/否问题链成，最终答案为这些问题答案的整数和。通过在Video-HopChain上训练Qwen3-VL-8B模型，提升了视频理解和推理基准的平均分数，从55.4提高到59.3。为了解决GRPO在处理困难和简单问题时的学习信号不足，提出了Confidence-Gated Exploration (CGE)方法，进一步提升了模型性能。

🔬 方法详解

问题定义：本文旨在解决视频推理中多跳问题的不足，现有方法在处理复杂推理时容易出现错误未被暴露的问题。

核心思路：提出Video-HopChain数据集，通过设计多跳问题链来增强模型的推理能力，并引入CGE方法以改善学习信号的不足。

技术框架：整体架构包括两个阶段：首先在标准视频数据集上训练Qwen3-VL-8B模型，然后在Video-HopChain上进行微调。CGE方法在此过程中用于优化学习信号。

关键创新：最重要的创新点在于引入了CGE方法，通过对问题的信心进行门控探索，解决了GRPO在处理困难和简单问题时的学习信号不足的问题。

关键设计：在CGE中，前4个回合的采样为常规方式，如果结果一致，则后4个回合使用策略中最有信心的token进行掩码处理，确保损失计算的有效性。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

实验结果显示，使用Video-HopChain数据集和CGE方法后，Qwen3-VL-8B模型在视频理解和推理基准上的平均分数从55.4提升至59.3，所有基准均有所改善，显示出显著的性能提升。

🎯 应用场景

该研究的潜在应用领域包括视频问答、视频内容分析和智能监控等。通过提升视频推理模型的性能，能够更好地支持自动化视频理解和分析，具有重要的实际价值和广泛的应用前景。

📄 摘要（原文）

HopChain has shown on still images that multi-hop data synthesis improves vision-language reasoning, because long chain-of-thought reasoning exposes errors that compound across steps, while most data used for reinforcement learning with verifiable rewards (RLVR) rarely demands a chain of visual evidence, so these weaknesses are likely to stay unexposed. We observe the same problem in video, where this framework has not yet been explored. We therefore build Video-HopChain, a dataset of 22,550 multi-hop video questions over 13,378 videos, together with a held-out benchmark of 1,000 questions. Each question chains three to six yes/no questions about moments in one video, and each yields one of two integers depending on its answer. The final answer is the sum of these integers, so an exact match on that sum gives the verifiable reward that RLVR needs. We first train Qwen3-VL-8B with GRPO on a standard video dataset, and a second stage on Video-HopChain then raises the mean over eight video understanding and reasoning benchmarks from 55.4 to 57.9 and improves every one of them. Training on such a dataset, however, exposes a known limitation of GRPO: its learning signal comes from the reward variance within a group, so hard questions whose rollouts are all incorrect and easy questions whose rollouts are all correct both leave the group with no gradient. To recover these groups at the same compute budget, we introduce Confidence-Gated Exploration (CGE). With 8 rollouts per question, CGE samples the first 4 as usual. If these 4 are either all correct or all incorrect, it samples the last 4 with the policy's most confident token masked inside the reasoning span, and removes the masked positions from the loss while all 8 rollouts enter the advantage. With CGE, the mean rises further to 59.3. We release the dataset, the checkpoint, and the data generation and training code.