Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior

📄 arXiv: 2310.11284v1 📥 PDF

作者: Ruibo Li, Chi Zhang, Zhe Wang, Chunhua Shen, Guosheng Lin

分类: cs.CV

发布日期: 2023-10-17

备注: An extension of our CVPR 2022 paper (RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior)

💡 一句话要点

提出自监督学习方法以解决3D场景流估计与运动预测问题

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics) 支柱七：动作重定向 (Motion Retargeting) 支柱八：物理动画 (Physics-based Animation)

关键词: 自监督学习 3D场景流 运动预测 点云处理 刚性运动估计 深度学习 计算机视觉

📋 核心要点

  1. 现有方法在3D场景流估计中依赖于标注数据，缺乏有效的自监督学习策略。
  2. 论文提出通过局部刚性运动估计生成伪标签，从而实现自监督学习，降低对标注数据的依赖。
  3. 在多个数据集上进行的实验表明，该方法在自监督场景流学习中表现优异，超越了部分监督学习方法。

📝 摘要（中文）

本文研究了基于点云的自监督3D场景流估计和类无关运动预测。通过将源点云分解为局部区域并假设每个区域为刚性体，利用刚性对齐生成伪场景流标签。为减少异常值影响，采用三步法进行刚性配准，确保生成的标签具有较高的置信度。实验结果表明，该方法在FlyingThings3D和KITTI数据集上实现了自监督场景流学习的新状态，且在nuScenes数据集上超越了以往的自监督方法。

🔬 方法详解

问题定义：本文旨在解决3D场景流估计和运动预测中的自监督学习问题。现有方法通常依赖于大量标注数据，限制了其应用范围。

核心思路：论文提出通过将源点云分解为局部区域并假设每个区域为刚性体，利用刚性对齐生成伪场景流标签。这种方法能够有效降低对标注数据的依赖。

技术框架：整体架构包括三个主要步骤：建立点对应关系、测量对应关系的置信度、基于对应关系及其置信度更新刚性变换。通过这些步骤，确保生成的伪标签具有较高的可信度。

关键创新：最重要的技术创新在于通过局部刚性运动估计生成伪标签，并引入置信度机制以减少异常值的影响。这与现有方法的显著区别在于不再依赖于真实标签。

关键设计：在参数设置上，采用了置信度加权的刚性配准方法，损失函数设计为结合了伪标签的置信度和区域一致性，网络结构则基于点云处理的最新进展进行优化。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在FlyingThings3D和KITTI数据集上，提出的方法在自监督场景流学习中达到了新的状态，超越了以往的最佳结果，且在nuScenes数据集上显著优于之前的自监督方法，展示了较强的性能提升。

🎯 应用场景

该研究具有广泛的应用潜力，尤其在自动驾驶、机器人导航和虚拟现实等领域。通过自监督学习，能够在缺乏标注数据的情况下，提升3D场景理解和动态物体识别的能力，推动相关技术的发展与应用。

📄 摘要（原文）

In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of the rigid motion of these individual parts. Building upon this observation, we propose to generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation, in which the source point cloud is decomposed into local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels. To mitigate the impact of potential outliers on label generation, when solving the rigid registration for each region, we alternately perform three steps: establishing point correspondences, measuring the confidence for the correspondences, and updating the rigid transformation based on the correspondences and their confidence. As a result, confident correspondences will dominate label generation and a validity mask will be derived for the generated pseudo labels. By using the pseudo labels together with their validity mask for supervision, models can be trained in a self-supervised manner. Extensive experiments on FlyingThings3D and KITTI datasets demonstrate that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even performing better than some supervised counterparts. Additionally, our method is further extended to class-agnostic motion prediction and significantly outperforms previous state-of-the-art self-supervised methods on nuScenes dataset.