Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning

📄 arXiv: 2609.24244v1 📥 PDF

作者: Santi Ram Tiwari, Nihal Naik, Devbrat Pandey, Nishant Sinha

分类: cs.CV, eess.IV

发布日期: 2026-09-21

备注: 6 pages, 4 figures, 4 tables

💡 一句话要点

提出一种无标签视觉证据信号以提升细粒度视觉语言推理

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态大型语言模型 视觉推理 无标签学习 对比证据差距 细粒度视觉问题 答案区域定位 模型信心

📋 核心要点

  1. 现有的多模态大型语言模型在细粒度视觉问题上表现不佳，主要由于视觉证据未被有效利用。
  2. 本文提出通过对比证据差距，利用无标签的视觉信号来定位答案区域，避免了昂贵的标注和训练成本。
  3. 实验结果显示，该方法在定位目标区域的精度上显著优于随机选择，并提高了细粒度推理的准确率。

📝 摘要（中文）

多模态大型语言模型（MLLMs）在细粒度视觉问题上表现不佳，主要是因为高分辨率图像在编码前被下采样，导致模型依赖语言先验而非视觉证据。本文提出了一种新方法，通过对比证据差距，利用模型对问题相关区域与无关区域的输出分布进行分析，发现答案所在的区域。实验表明，该方法在不需要标注的情况下，能够将目标区域的定位精度提高4.4到5.1倍，并将细粒度准确率从70%提升至85%。此外，结合模型自身信心的信号可以更好地预测答案的正确性，AUC值高达0.99。

🔬 方法详解

问题定义：本文旨在解决多模态大型语言模型在细粒度视觉问题上因视觉证据缺失而导致的推理不足。现有方法依赖昂贵的标注数据或外部教师，限制了其应用。

核心思路：提出通过对比证据差距，利用模型在问题相关区域与无关区域的输出分布，来无标签地发现答案区域。这种方法不需要额外的训练或标注，降低了成本。

技术框架：整体流程包括输入高分辨率图像，计算模型在不同区域的输出分布，选择输出分布最集中的区域作为答案区域。该方法依赖于单视图的无标签标准进行区域选择。

关键创新：最重要的创新在于提出了对比证据差距的概念，利用这一信号来定位答案区域，与传统方法相比，避免了对标注数据的依赖。

关键设计：在实验中，采用了单视图的无标签标准进行区域选择，确保了方法的简洁性和有效性。实验结果表明，该方法在定位精度和细粒度推理准确率上均有显著提升。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，该方法在定位目标区域的精度上显著优于随机选择，精度提升4.4到5.1倍，细粒度推理的准确率从70%提升至85%。结合模型信心的信号，AUC值高达0.99，能够有效识别错误答案。

🎯 应用场景

该研究的潜在应用领域包括智能问答系统、视觉搜索引擎和自动图像标注等。通过提升细粒度视觉推理能力，可以在多模态交互中实现更高的准确性和用户体验，未来可能推动相关技术的广泛应用。

📄 摘要（原文）

Multimodal large language models (MLLMs) fail at fine-grained visual questions less because they cannot reason than because they never see the evidence: high-resolution images are downsampled before encoding, so the model answers from linguistic priors. The standard remedies are expensive: annotated answers (SFT), hand-engineered verifiers (RLVR), or a large external teacher (on-policy distillation). We ask whether the visual evidence itself can supply the signal for free. We formalize the contrastive evidence gap, the per-token log-likelihood ratio that a model assigns to its own output when conditioned on a question-relevant region versus an irrelevant one, and study it across Qwen2.5-VL-7B, Qwen3-VL-8B, and Qwen3-VL-30B-A3B on V*Bench. Our main positive result is training-free: selecting the candidate crop under which the model's answer distribution is most peaked, using a single-view, label-free criterion, discovers the answer-bearing region with no bounding boxes, training, or labels. It localizes the target 4.4 to 5.1 times better than chance and raises fine-grained accuracy from 70 percent to 85 percent at inference. We further show that the gap is complementary to the model's own confidence. Combining them predicts correctness better than either alone, with AUC up to 0.99, and flags confidently wrong answers, with AUC ranging from 0.97 to 1.00 within the high-confidence subset. All effects concentrate on perception-bottleneck questions and vanish on a global-context control. Finally, we report an honest negative result: converting the same signal into a training method, gated self-distillation (SEG-Distill), does not outperform the base model at pilot scale across three gate designs, while more aggressive gating degrades accuracy. The signal is real, but converting it into training gains remains an open problem.