Title: SimuScene: Generating Compositional 3D Scenes Ready for Simulation from a Single Image

Original: arXiv:2606.03994v1 Announcement Type: New

Abstract:

The ability to reconstruct interactive, simulation-ready 3D environments from a single photograph remains a significant hurdle in the field of robotic manipulation. Although contemporary single-image reconstruction techniques can generate plausible shapes for individual objects, assembling these elements into a cohesive scene often results in physical instability; objects may interpenetrate, hover, or sink, causing the simulation to fail. Current approaches that incorporate physics typically treat this issue as a secondary layout correction step, which fails to rectify the fundamental geometric inaccuracies present in the initial reconstruction.

To overcome these limitations, we present SimuScene, a novel pipeline for compositional 3D reconstruction that integrates physics directly into the processes of shape and layout estimation. Instead of employing physics engines solely for post-hoc layout refinement, we leverage them as diagnostic tools throughout the generative phase. By simulating the reconstructed objects under gravitational forces, we transform instances of penetration and support failure into measurable correction signals. These signals guide the system to adjust the gravity-axis dimensions and resample amodal shapes. This physics-informed feedback mechanism effectively reduces the accumulation of reconstruction errors, resulting in a stable, compositionally sound 3D scene that is ready for simulation. Our extensive experimental evaluations show that SimuScene achieves state-of-the-art results in benchmarks for geometric alignment and physical stability. Furthermore, we demonstrate the practical value of SimuScene by utilizing the reconstructed environments in tasks involving humanoid control and robot-arm manipulation.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC