Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving

📄 arXiv: 2610.25734v1 📥 PDF

作者: Jie Wang, Guang Li, Zhijian Huang, Jinlong Li, Chenxu Dang, Hangjun Ye, Yahong Han, Long Chen

分类: cs.CV, cs.RO

发布日期: 2026-10-19

备注: NeurIPS 2026

💡 一句话要点

提出LECDrive以解决自主驾驶中的稀疏监督问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱三：空间感知与语义 (Perception & Semantics) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 自主驾驶 视觉-语言-动作 潜在专家认知链 密集知识内化 课程学习 多模态融合

📋 核心要点

  1. 现有的VLA模型在自主驾驶中面临稀疏监督的问题，导致重要场景信息的丢失。
  2. 本文提出LECDrive，通过潜在专家认知链的方式，促进密集世界模型的构建。
  3. 实验结果显示，LECDrive在物体感知、状态预测和轨迹规划等任务上显著提升了性能。

📝 摘要（中文）

自主驾驶需要对视觉、语义和几何信息进行密集理解，但现有的视觉-语言-动作（VLA）模型通常依赖稀疏监督，如语言指令或轨迹信号。这种从密集视觉观测到稀疏表示的直接映射，导致模型压缩或丢弃对安全驾驶至关重要的场景信息。为了解决这一问题，本文提出了一种新范式“Think Densely, Act Sparsely”，并通过LECDrive（潜在专家认知链）实现。该方法通过嵌入逐步的潜在专家认知链，模仿人类的层次感知过程，提升了模型对密集知识的内化能力。实验表明，LECDrive在多个自主驾驶基准测试中显著提升了基础VLM的性能。

🔬 方法详解

问题定义：本文旨在解决现有自主驾驶模型在稀疏监督下对重要场景信息的压缩和丢失问题。现有方法无法有效利用密集的视觉信息，导致安全驾驶的潜在风险。

核心思路：论文提出“Think Densely, Act Sparsely”的新范式，通过引入潜在专家认知链，模仿人类的层次感知过程，促进模型对密集知识的内化。

技术框架：LECDrive框架包括多个模块，首先注入任务特定的潜在标记作为载体，然后通过逐步的课程学习结构化监督，从表示到语义和几何信息进行训练。

关键创新：最重要的创新在于构建了一个层次化的潜在认知链，使得模型能够在保持高效推理的同时，内化来自基础视觉模型的密集知识。这一设计与传统的稀疏监督方法有本质区别。

关键设计：在模型设计中，采用了紧凑的任务特定潜在标记，结合DINOv3、SAM3和DepthAnything3等基础视觉模型，确保了视觉表示、语义结构和空间几何信息的有效融合。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，LECDrive在多个自主驾驶基准测试中均显著提升了基础VLM的性能，尤其在物体感知、状态预测和轨迹规划任务上，提升幅度达到XX%（具体数据待补充）。

🎯 应用场景

该研究的潜在应用领域包括自动驾驶、智能交通系统和机器人导航等。通过构建密集的世界模型，LECDrive能够提升自主驾驶系统的安全性和可靠性，未来可能在智能城市和无人驾驶技术中发挥重要作用。

📄 摘要（原文）

Autonomous driving fundamentally demands dense understanding of visual, semantic, and geometric information, yet existing Vision-Language-Action (VLA) models are typically trained with sparse supervision such as language instructions or trajectory signals. This direct mapping from dense visual observations to sparse representations forces the model to compress or even discard critical scene information essential for safe driving. We argue that this limitation does not stem from insufficient model capacity, but from the lack of explicit mechanisms that encourage the formation of dense world representations in the latent space. In this spirit, we propose a novel paradigm for driving intelligence, Think Densely, Act Sparsely, and instantiate it with an efficient and interpretable framework, LECDrive (Latent Expert Cognitive Chains for Vision-Language-Action autonomous Driving). The core idea is to mimic the hierarchical perception process of humans by embedding a progressive chain of latent expert cognition within the VLA model. Concretely, we inject a compact set of task-specific latent tokens as carriers, enabling the model to internalize dense knowledge from foundational vision models, including visual representations (DINOv3), semantic structures (SAM3), and spatial geometry (DepthAnything3), while maintaining efficient inference. This interdependent cognitive chain follows a curriculum learning principle, where supervision is progressively structured from representation to semantics and geometry. Extensive experiments demonstrate that LECDrive consistently improves the performance of base VLMs across multiple autonomous driving benchmarks, covering key tasks such as object perception, state prediction, and trajectory planning. These findings suggest that constructing dense world models through structured latent expert chains is a promising direction for VLA-based autonomous driving.