GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

📄 arXiv: 2609.39601v1 📥 PDF

作者: Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, Yi Zou, Kaixuan Wang, Tianxing Chen, Wenxuan Song, Bohan Zhou, Mingleyang Li, Siqiao Huang, Yuqi Ye, Caigao Jiang, Wei Wei, Ruihai Wu, Hang Zhang, Yixiao Ge, Shuchang Zhou, Shilong Liu, Xianming Liu, Ping Luo, Shiyu Huang

分类: cs.CV, cs.AI, cs.LG, cs.RO

发布日期: 2026-09-30

备注: 64 pages, including supplementary material. Project page: https://groundingpi.github.io/ Code: https://github.com/groundingpi/GroundingPI Model: https://huggingface.co/GroundingPI/GroundingPI

💡 一句话要点

提出GroundingPI以解决物理智能中的精确定位问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 物体定位 视觉-语言模型 多模态学习 强化学习 机器人操作 自动驾驶 基础模型 物理智能

📋 核心要点

  1. 现有的视觉-语言-动作模型在复杂环境中对物体的精确定位能力不足，尤其是对于小物体。
  2. GroundingPI通过生成量化坐标的点和框，结合多模态预训练和强化学习，提供了一种新的解决方案。
  3. 在多个基准测试中，GroundingPI的表现超越了现有的主流模型，尤其在机器人操作和自动驾驶任务中显著提升了性能。

📝 摘要（中文）

精确的物体定位至关重要，尤其是在复杂环境中和小物体的情况下。现有的视觉-语言-动作模型和世界-动作模型在这些场景中表现不佳。本文提出了GroundingPI，一个具有4B参数的基础模型，能够生成量化坐标的点和框。通过多模态和空间预训练、监督微调及强化学习等方法进行训练，GroundingPI在34个基准测试中超越了44个基线，平均达到73.68%的准确率，优于更大的GPT-6 Astra。该模型在机器人操作和自动驾驶等下游任务中表现出色，显示了其作为物理智能基础的潜力。

🔬 方法详解

问题定义：本文旨在解决物体在复杂环境中的精确定位问题。现有的视觉-语言-动作模型和世界-动作模型在处理小物体和杂乱场景时表现不佳，无法满足闭环控制的需求。

核心思路：GroundingPI模型通过生成量化的坐标点和框，利用共享词汇进行物体定位。该模型结合了多模态和空间预训练、监督微调及强化学习，旨在提升物体定位的准确性和速度。

技术框架：GroundingPI的训练流程包括多个阶段：首先进行多模态和空间预训练，然后进行监督微调，最后通过强化学习优化模型性能。模型的训练数据来源于公共数据集和专用数据引擎。

关键创新：GroundingPI的主要创新在于其生成的量化坐标点和框的设计，使得模型在复杂环境中能够更准确地进行物体定位。这一方法与传统的视觉-语言模型有本质区别，后者通常依赖于通用的视觉-语言和视频生成骨干网络。

关键设计：在模型设计中，GroundingPI使用了4B参数，结合了多模态数据的丰富性和空间信息的准确性。损失函数的设计考虑了定位精度和速度的平衡，确保模型在实际应用中的有效性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

GroundingPI在34个基准测试中超越了44个基线，平均准确率达到73.68%，优于更大的GPT-6 Astra（71.54%）。在RoboTwin 2.0上，GroundingPI在所有四个分布外设置中均超越了主流骨干网络，提升幅度高达24.8%。在nuScenes数据集上，GroundingPI的平均开环L2误差为0.296米，显示出其卓越的性能。

🎯 应用场景

GroundingPI在机器人操作和自动驾驶等领域具有广泛的应用潜力。其精确的物体定位能力能够显著提升机器人在复杂环境中的操作效率，同时为自动驾驶系统提供更可靠的感知基础。未来，该模型可能在智能家居、无人机导航等领域发挥重要作用。

📄 摘要（原文）

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.