Spheriverse: 3D Scene Understanding from Spherical Observations in the Wild

📄 arXiv: 2609.09012v1 📥 PDF

作者: Fei Teng, Sheng Wu, Mengfei Duan, Guoqiang Zhao, Junhui Ma, Kai Luo, Siyu Li, Hao Shi, Zhiyong Li, Kailun Yang

分类: cs.CV, cs.RO, eess.IV

发布日期: 2026-09-08

备注: The established benchmark and source code will be available at https://feit-feiteng.github.io/Spheriverse

🔗 代码/项目: PROJECT_PAGE

💡 一句话要点

提出Spheriverse以解决3D场景理解中的球面观测问题

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics)

关键词: 3D场景理解 球面观测 语义映射 自动驾驶 机器人导航 深度学习 数据集

📋 核心要点

  1. 现有方法在处理球面观测与笛卡尔坐标之间的表示差距时面临挑战，导致几何对应和语义聚合困难。
  2. 论文提出Spheriverse数据集及SphereOcc框架，通过球面几何建模与语义证据检索的结合来解决上述问题。
  3. SphereOcc在多个场景中表现优异，mIoU达到13.91%，GeoIoU达到24.65%，超越了现有最佳方法TPVFormer和SurroundOcc。

📝 摘要（中文）

球面观测为3D场景理解提供了全球视觉上下文。然而，视觉信息以角度域编码，而物理世界则以笛卡尔坐标表示，这种跨空间表示的差距使得几何对应和语义证据聚合变得复杂。为此，本文引入Spheriverse数据集，包含64,400个时间对齐的球面图像-LiDAR配对，涵盖多样的场景、光照和天气条件，并提供细粒度的语义类别。我们建立了语义占用预测、语义映射和3D物体检测的基准，评估了30多种方法的整体和场景比较。为实现密集预测，我们提出了SphereOcc框架，将球面几何建模与语义证据检索相结合。

🔬 方法详解

问题定义：本文旨在解决球面观测与笛卡尔坐标之间的表示差距，现有方法在几何对应和语义证据聚合上存在不足。

核心思路：通过引入Spheriverse数据集和SphereOcc框架，结合球面几何建模与语义证据检索，来有效地处理3D场景理解中的复杂性。

技术框架：SphereOcc框架主要包括两个模块：Cartesian-Spherical Representation Remodeling (CSRR)和Spherical Evidence Re-querying (SER)。CSRR将球面范围-方位几何整合到笛卡尔体素特征中，而SER则根据体素内容和几何信息自适应检索相关的语义证据。

关键创新：最重要的创新在于CSRR和SER的结合，使得球面几何信息能够有效地融入到笛卡尔坐标系中，从而提升了语义理解的准确性。

关键设计：在SphereOcc中，采用了区域调制的方式来处理体素特征，并设计了特定的损失函数以优化模型在不同场景下的表现。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

SphereOcc在多个场景中表现出色，mIoU达到13.91%，GeoIoU达到24.65%，分别超越了TPVFormer和SurroundOcc的最佳表现1.70和2.10个百分点，显示出在不同空间划分和视场条件下的持续优势。

🎯 应用场景

该研究在自动驾驶、机器人导航和增强现实等领域具有广泛的应用潜力。通过提升3D场景理解的准确性，能够为智能系统提供更为丰富的环境感知能力，从而推动相关技术的发展与应用。

📄 摘要（原文）

Spherical observations provide global visual context for 3D scene understanding. However, visual information is encoded in an angular domain, whereas the physical world is represented in Cartesian coordinates. This cross-space representation gap complicates geometric correspondence and semantic evidence aggregation. To delve into this challenge, we introduce Spheriverse, comprising $64,400$ temporally aligned spherical image-LiDAR pairs organized into 644 sequences. The dataset spans diverse scenes, illumination, and weather conditions, with fine-grained semantic classes. We further establish benchmarks for semantic occupancy prediction, semantic mapping, and 3D object detection, evaluating 30+ methods through overall and scene-wise comparisons. For dense prediction, we propose SphereOcc, an occupancy framework that couples spherical geometry modeling with semantic evidence retrieval. Cartesian-Spherical Representation Remodeling (CSRR) incorporates spherical range-azimuth geometry into Cartesian voxel features through region-wise modulation. Spherical Evidence Re-querying (SER) then conditions queries on voxel content and range-height-azimuth geometry to adaptively retrieve relevant semantic evidence from source spherical image features. SphereOcc achieves 13.91% mIoU and 24.65% GeoIoU, outperforming the respective best-performing methods, TPVFormer and SurroundOcc, by 1.70 and 2.10 percentage points. It also ranks first in both metrics across all five scenes, with consistent advantages across the evaluated spatial partitions and reduced fields of view. The established benchmark and source code will be available at https://feit-feiteng.github.io/Spheriverse.