Learning to Think and Explore in 3D World

📄 arXiv: 2610.21783v1 📥 PDF

作者: Yijiang Li, Bingyang Wang, Nuno Vasconcelos

分类: cs.CV

发布日期: 2026-10-15

💡 一句话要点

提出ACT3D以解决3D空间中的视觉语言模型探索问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱六：视频提取与匹配 (Video Extraction)

关键词: 视觉语言模型 3D探索 强化学习 自主学习 虚拟现实 机器人导航 决策支持

📋 核心要点

  1. 现有的视觉语言模型在3D空间探索中面临多视图输入的限制，无法有效模拟人类的视觉处理方式。
  2. 本文提出的ACT3D模型通过单张图像进行推理和导航，利用强化学习自主学习有效的探索策略。
  3. 实验结果显示，ACT3D在多个基准测试中表现优异，准确率显著提升，并在未见领域中也有良好泛化能力。

📝 摘要（中文）

本研究探讨了视觉语言模型（VLMs）如何在3D空间中学习思考和主动探索。与传统方法不同，本文仅使用单张图像作为输入，模拟人类的视觉处理方式，模型通过与环境的交互（如移动和观察）进行推理和导航。经过初步的轨迹监督微调后，模型采用强化学习（RL）进行训练，奖励主要基于最终答案的正确性。实验结果表明，ACT3D在六个3D基准测试中平均准确率达到53.2%，较基础模型Qwen提升22.0%，并超越了现有的商业模型Gemini 3.6 Flash。

🔬 方法详解

问题定义：本研究旨在解决视觉语言模型在3D空间中探索的有效性问题。现有方法通常依赖多视图输入，限制了模型的灵活性和人类般的视觉处理能力。

核心思路：ACT3D模型通过单张图像输入，模拟人类的视觉探索过程，采用强化学习策略使模型在与环境交互中自主学习探索行为。

技术框架：模型的整体架构包括初始的轨迹监督微调阶段，随后进入强化学习阶段。模型从第一人称视角开始，接收视觉问题，通过推理和导航调整视角，最终给出答案。

关键创新：ACT3D的主要创新在于其使用单视图输入和无监督的强化学习策略，使得模型能够自主学习何时及如何进行有效的探索，这与传统方法形成鲜明对比。

关键设计：在训练过程中，模型的奖励机制主要基于最终答案的正确性，而非对视角或探索行为的直接监督。这种设计使得模型能够灵活适应不同的探索需求。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

ACT3D在六个3D基准测试中实现了53.2%的平均准确率，较基础模型Qwen提升22.0%。此外，该模型在使用50-70%更少输入帧的情况下，超越了商业模型Gemini 3.6 Flash，并在未见领域中表现出5.9%的泛化提升。

🎯 应用场景

该研究的潜在应用领域包括虚拟现实、机器人导航和自动驾驶等场景。通过提升模型在3D环境中的探索能力，ACT3D可以为智能体提供更为精准的决策支持，推动相关技术的发展和应用。

📄 摘要（原文）

We study how vision-language models (VLMs) can learn to think and actively explore in 3D space. Instead of providing multi-view images, videos, or 3D representations to the model, we preserve the native visual encoding of VLMs, i.e., a single image, one at a time, just like humans, and treat the 3D representation only as an environment with which the model can interact through actions (e.g., move left) and observations (rendered images). Each rollout begins from an initial first-person view and a visual question. The model can then iteratively reason, navigate, and reorient its viewpoint before committing to a final answer. Following an initial trajectory-supervised finetuning, we apply reinforcement learning (RL) with rewards driven primarily by final-answer correctness. Crucially, RL provides no direct supervision over which viewpoints the model should visit or which exploration actions it should take. Instead, the model must learn an effective exploration policy solely from the extent to which its interactions improve performance. Across six 3D benchmarks, our model ACT3D achieves an average accuracy of 53.2% that improves its base Qwen model by 22.0% and surpasses proprietary Gemini 3.6 Flash. ACT3D also outperforms existing open-source 3D methods while using 50-70% fewer input frames, and generalizes to unseen domains with an average improvement of 5.9% over its supervised initialization. Further analysis reveals adaptive, question-conditioned exploration. Performance gains over the base model increase from 0.6% on questions answerable from the initial view to 16.3% on those requiring extensive exploration. ACT3D searches for occluded targets, avoids unnecessary exploration, and acquires complementary viewpoints for spatial comparisons. These findings demonstrate that outcome-based RL enables VLMs not only to reason over visual evidence, but also to learn when and how to acquire it.