Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error
作者: Iván Verdugo Guerra, Ezequiel López Rubio, Jorge García González
分类: cs.CV
发布日期: 2026-10-06
备注: 18 pages, 11 figures, 9 tables. Code: https://github.com/ivanver02/semantic-lifting-3dgs
💡 一句话要点
提出后训练语义提升方法以解决3D高斯点云分类问题
🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics)
关键词: 3D高斯点云 后训练提升 多视角信息融合 物体检测 语义分割 深度学习 计算机视觉
📋 核心要点
- 现有的3D高斯点云分类方法在不同视角下的分类结果不一致，且受到遮挡和检测器置信度的影响。
- 本文提出了一种后训练提升方法，针对单一目标类，结合多个视角的信息，权重依据高斯的可见性。
- 实验结果显示，该方法在多个场景中提升了mIoU值，尤其在ScanNet++测试场景中，较之前版本提高了0.24。
📝 摘要（中文）
在3D高斯点云模型中，同一高斯在多个视角下的分类结果可能不一致，且由于遮挡和检测器置信度的差异，导致训练结果不稳定。本文提出了一种后训练提升方法，针对单一目标类，结合来自多个视角的信息，权重由每个高斯在视角中的可见性决定。通过设定两个阈值，筛选出高置信度的种子，并扩展其周围的连通组件。实验结果表明，该方法在多个场景中有效提升了mIoU值，尤其在ScanNet++测试场景中，较之前版本提高了0.24，且能够统一适用于所有类和场景。
🔬 方法详解
问题定义：本文旨在解决3D高斯点云模型中，由于视角差异导致的分类不一致性问题。现有方法在不同视角下的检测结果不稳定，且无法有效整合多视角信息。
核心思路：提出的后训练提升方法通过单一目标类的信息整合，结合来自多个视角的证据，利用高斯的可见性进行加权，从而提高分类的准确性。
技术框架：整体流程包括三个主要阶段：首先，收集来自不同视角的目标和非目标证据；其次，通过设定两个阈值筛选高置信度的种子并扩展连通组件；最后，将标签从高斯转移到可见且已标注的网格顶点。
关键创新：该方法的创新之处在于能够有效分离来自检测器、提升过程和表示转移的三种误差源，并通过统一的阈值处理不同类和场景，提升了模型的鲁棒性。
关键设计：在参数设置上，采用了主阈值β和次阈值γβ来筛选高置信度种子和扩展连通组件，实验中在七个Replica验证场景上选择了这些阈值，并在十个ScanNet++测试场景上进行了评估。实验结果显示，使用YOLO掩码时，mIoU值达到了0.54，整体性能显著提升。
🖼️ 关键图片
📊 实验亮点
实验结果表明，本文方法在验证场景上的平均mIoU达到了0.93，而在ScanNet++测试场景上为0.80，相较于之前版本提高了0.24，显示出显著的性能提升。该方法能够统一适用于所有类和场景，具有较强的实用性。
🎯 应用场景
该研究的潜在应用领域包括自动驾驶、机器人视觉和增强现实等场景，能够有效提升3D物体识别的准确性和鲁棒性。未来，该方法有望在多视角数据融合和实时物体检测中发挥重要作用，推动相关技术的发展。
📄 摘要（原文）
The same Gaussian of a 3D Gaussian Splatting model is seen from many views, and these views do not always agree on the class it belongs to. The Gaussian may be occluded in some of them, and the confidence of the detector is not the same from one view to another. The ground truth, on the other hand, is given as an annotated mesh, because two training runs do not produce the same Gaussians. In this work, we propose a post-training lifting method that works with one target class at a time and combines the information coming from all the views. Target and non-target evidence are accumulated simultaneously, weighted by the visibility of each Gaussian in each view. After that, the Gaussians are filtered with two thresholds: a main threshold $β$ selects the high-confidence seeds, and a lower one $γβ$ adds the connected components around them. For the evaluation, the labels are transferred from the Gaussians to the mesh vertices that are both visible and annotated. With this design, we can separate three sources of error: the 2D detector, the lifting and the transfer between representations. The thresholds and the transfer operator are chosen on seven Replica validation scenes, and the method is evaluated on ten held-out ScanNet++ scenes with the same values for every scene and class. The mean mIoU on the validation scenes was 0.93 with masks from the dataset annotations and 0.65 with YOLO masks, and on the ScanNet++ test scenes it was 0.80 and 0.54. Compared with thresholding the evidence per view, as a previous version of the method did, the fraction improves the test mIoU by 0.24 and makes it possible to use a single threshold for all the classes and scenes of both datasets. Finally, the error analysis shows that most of the remaining error comes from the detector.