Title: GS-ROR$^2$: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstruction

Original: arXiv:2406.18544v4 Announce Type: replace

Abstract: 3D Gaussian Splatting (3DGS) has demonstrated strong potential in novel view synthesis, owing to its capacity for detailed expression and rapid rendering performance. However, generating relightable 3D assets and achieving accurate geometry reconstruction using 3DGS remains challenging, especially for reflective surfaces, because its discontinuous nature complicates geometric constraints. While Volumetric Signed Distance Field (SDF) techniques offer reliable geometry reconstruction, their reliance on costly ray marching impedes real-time usage and prolongs training times. Furthermore, these methods often fail to capture fine geometric details. To address these issues, we introduce a bidirectional, complementary guidance framework between 3DGS and SDF. This approach incorporates SDF-aided Gaussian splatting to facilitate efficient optimization of the relighting model, alongside GS-guided SDF enhancement to ensure high-fidelity geometry reconstruction. Central to our SDF-aided Gaussian splatting is the mutual supervision of depth and normal information between blended Gaussians and the SDF, which eliminates the need for expensive volume rendering of the SDF. This mutual supervision allows the learned blended Gaussians to be tightly constrained with minimal computational overhead. Since the Gaussians are rendered using deferred shading, the resulting alpha-blended images appear smooth, yet individual Gaussians may act as outliers, leading to floating artifacts. To mitigate this, we propose an SDF-aware pruning strategy that removes Gaussian outliers situated far from the SDF-defined surface, thereby eliminating floaters. Consequently, our Gaussian Splatting (GS) framework yields plausible normals and realistic relighting results, although the mesh derived from depth alone remains suboptimal. To resolve this, we design a GS-guided SDF refinement process that leverages the blended normals from the Gaussians to fine-tune the SDF. This enhancement enables our method to generate high-quality meshes for reflective objects, incurring only an additional 17% in training time.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC