PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving

📄 arXiv: 2607.20175v1 📥 PDF

作者: Yushan Liu, Tianxiong Lv, Bohua Wang, Hangqi Fan, Chenxu Zhao, He Zheng, Xuchang Zhong, Yifan Xie, Congyang Zhao, Zhihao Liao, Leigang Luo, Yang Cai, Xiao-Ping Zhang, Wenbo Ding

分类: cs.CV

发布日期: 2026-07-22

💡 一句话要点

提出PerceptDrive以解决自主驾驶中的感知优先世界-动作建模问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 自主驾驶 感知建模 专家路由 动态场景 轨迹生成 蒸馏训练

📋 核心要点

  1. 现有方法在自主驾驶中面临感知信息利用不足和动态场景适应性差的挑战。
  2. 论文提出PerceptDrive框架，通过自适应专家路由和先验保留来优化世界-动作建模过程。
  3. 实验结果显示，PerceptDrive在多个基准测试中表现优异，显著提升了规划性能。

📝 摘要（中文）

冻结的感知基础模型能够编码丰富的几何、语义和动态知识。然而，狭窄的条件接口可能会削弱任务相关线索，而静态融合无法根据每个场景调整专家的贡献。为此，本文提出了PerceptDrive，一个具有自适应专家路由的感知优先世界-动作建模框架。该框架将来自冻结的、适应驾驶的提供者的教师蒸馏先验与来自冻结的自监督视频编码器的密集观察潜在变量输入到一个可训练的专家路由世界-动作模型中。实验结果表明，PerceptDrive在NAVSIM v1和NAVSIM v2上分别达到了90.4和90.2的PDMS，超越了现有方法。

🔬 方法详解

问题定义：本文旨在解决自主驾驶中感知优先世界-动作建模的挑战，现有方法在动态场景下无法有效利用感知信息，导致规划性能不足。

核心思路：PerceptDrive通过引入自适应专家路由机制，结合教师蒸馏先验和自监督学习的潜在变量，增强了模型对场景的适应能力和规划精度。

技术框架：该框架包括多个模块：首先，冻结的感知模型提供教师蒸馏先验；其次，密集观察潜在变量由自监督视频编码器生成；最后，专家路由模型根据共享场景表示生成软门控，结合专家条件进行轨迹生成。

关键创新：PerceptDrive的创新在于自适应路由机制和先验保留目标，使得每个专家分支能够有效利用其特定的先验信息，从而提升了规划的直接性和准确性。

关键设计：模型设计中采用了基于规则的子度量估计作为分支特定轨迹草案的软门控蒸馏目标，损失函数中包含了先验保留目标，以确保每个专家分支的有效性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，PerceptDrive在NAVSIM v1和NAVSIM v2上分别达到了90.4和90.2的PDMS，显著优于现有方法，验证了自适应专家路由和先验保留的有效性，提升幅度明显。

🎯 应用场景

PerceptDrive的研究成果在自动驾驶领域具有广泛的应用潜力，能够提升自主驾驶系统在复杂动态环境中的决策能力。未来，该框架可扩展到其他需要实时感知与决策的机器人系统中，推动智能交通和自动化技术的发展。

📄 摘要（原文）

Frozen perception foundation models encode rich geometric, semantic, and dynamic knowledge. Yet narrow conditioning interfaces may attenuate task-relevant cues, while static fusion cannot adjust expert contributions to each scene. We cast this challenge as the prior-to-plan transfer problem and introduce PerceptDrive, a perception prior world-action modeling framework with adaptive expert routing. PerceptDrive feeds teacher-distilled priors from a frozen, driving-adapted provider and dense observation latents from a frozen self-supervised video encoder into a trainable expert-routed world-action model. Expert-specific query branches process these signals, while a prior-retention objective anchors each branch to its prior. A router predicts soft gates from a shared scene representation and combines the expert conditions before trajectory generation. During training, privileged rule-based sub-metric estimates for branch-specific trajectory drafts provide soft-gate distillation targets. The predicted action-free future latent conditions a flow-matching actor. At inference, privileged components are absent; with one front-facing camera, PerceptDrive generates one trajectory per planning step without test-time scoring, reranking, or search. Experiments show that PerceptDrive achieves state-of-the-art performance with 90.4 PDMS on NAVSIM v1 and 90.2 EPDMS on NAVSIM v2, outperforming existing methods. Ablations confirm complementary gains from prior retention and scene-conditioned routing, alongside differential reliance on the three priors. These results demonstrate that preserving and adaptively routing perception priors improves direct planning without test-time candidate selection.