HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis

📄 arXiv: 2609.26793v1 📥 PDF

作者: Shufan Sun, Chen Wang, Enxin Song, Jiatao Gu, Lingjie Liu

分类: cs.CV

发布日期: 2026-09-22

备注: Project Page: http://cwchenwang.github.io/harmony

💡 一句话要点

提出HARMONY以解决单目图像到场景合成的挑战

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics) 支柱七：动作重定向 (Motion Retargeting) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 单目图像重建 3D场景合成 代理推理 视觉几何 层次化框架 深度学习 室内场景

📋 核心要点

  1. 现有方法在从单目图像重建3D场景时，无法同时实现高保真度和准确的物体间关系。
  2. HARMONY通过层次化的思维链框架，结合代理推理与视觉几何基础，逐步恢复3D场景布局。
  3. 实验结果显示，HARMONY在合成和真实图像上均超越了现有重建方法，提供更一致的语义和更高的细节保留。

📝 摘要（中文）

近年来，复合3D场景重建主要从两个方向探索：代理推理提供空间关系的语义理解，但与输入图像的精确对齐不足；而视觉几何基础模型从输入图像预测密集点图，但重建质量有限。因此，从单个单目图像恢复完整3D场景，准确捕捉物体间关系和高保真重建质量仍然具有挑战性。本文提出HARMONY，一个层次化的思维链框架，结合代理推理和视觉几何基础。HARMONY从空的3D平面图开始，通过校准相机建立语义基础空间框架，利用代理VLM推理恢复3D房间布局及初始放置顺序，并以层次顺序放置物体。实验表明，HARMONY在合成和真实图像上均优于现有重建基线。

🔬 方法详解

问题定义：本文旨在解决从单个单目图像恢复完整3D场景的问题。现有方法在空间关系的语义理解与图像精确对齐之间存在不足，导致重建质量不高。

核心思路：HARMONY通过层次化的思维链框架，结合代理推理与视觉几何基础，逐步恢复3D场景布局，从而实现更高的重建质量和准确性。

技术框架：HARMONY的整体架构包括几个主要模块：首先，通过校准相机建立语义基础空间框架；然后，利用代理VLM推理恢复3D房间布局及初始放置顺序；接着，按照层次顺序放置物体，并进行几何基础的细化。

关键创新：HARMONY的关键创新在于结合了代理推理与视觉几何基础，通过层次化的物体放置策略，避免了错误的累积，提升了重建的准确性和一致性。

关键设计：在设计中，采用深度优先遍历策略进行家具放置，确保每个放置都基于之前解析的结构，同时引入反馈循环以减少错误累积。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

HARMONY在合成和真实图像上的实验结果显示，其重建质量显著优于现有基线，具体表现为更一致的语义和更高的细节保留。与GPT-6 Astra的定性比较表明，HARMONY在物体排列的忠实度和场景细节的保留上表现更佳。

🎯 应用场景

该研究的潜在应用领域包括室内设计、虚拟现实和游戏开发等。通过提供高保真的3D场景重建，HARMONY能够帮助设计师和开发者更高效地创建和优化虚拟环境，提升用户体验。未来，该技术可能推动智能家居、建筑可视化等领域的发展。

📄 摘要（原文）

Compositional 3D scene reconstruction has recently been explored from two directions: agentic reasoning that provides semantic understanding of spatial relationships but lacks precise alignment with input images; and visual geometry foundation models that predict dense point maps from input images but the reconstruction quality is limited. Therefore, recovering a complete 3D scene from a single monocular image with accurate inter-object relationships and high-fidelity reconstruction quality remains challenging. In this paper, we present HARMONY, a hierarchical chain-of-thought framework that leverages both agentic reasoning and visual geometry foundation. Given an image of an indoor scene, starting from an empty 3D floorplan, HARMONY first calibrates the camera against the reference image to establish a semantically-grounded spatial frame, then uses agentic VLM reasoning to recover the 3D room layout and an initial placement order. It then places the objects in a hierarchical order, from wall-mounted elements, free-standing furniture, to dependent decorations on top of furniture. We also use depth-first traversal for furniture so each placement conditions on previously resolved structure and a reflective feedback loop to avoid error accumulation. After each object placement by VLM, we use the point cloud estimations to perform geometry-based refinement so that the rendered image aligns better with the input. HARMONY can produce 3D scenes that are semantically consistent and perceptually aligned with the reference image, extending single-image compositional reconstruction to complex indoor scene images. Experiments on synthetic and real-world images demonstrate that HARMONY outperforms the evaluated reconstruction baselines, while qualitative comparisons with GPT-6 Astra suggest more faithful object arrangements and better preservation of scene details.