Title: PersistGS: Enforcing Object Permanence in 4D Gaussian Splatting via Differentiable Physics

Abstract:

Standard Dynamic 3D Gaussian Splatting (3DGS) techniques reconstruct scenes that evolve over time by leveraging photometric supervision on synchronized multi-camera video footage. However, a significant limitation arises when a moving object is completely hidden from all training viewpoints: the supervision signal disappears, causing the Gaussians representing the object to lose gradient updates and subsequently degrade. Current solutions for handling partial observations in neural reconstruction typically depend on learned generative priors, which tend to favor visual realism at the expense of physical accuracy.

To address this, we introduce PersistGS, a novel framework that maintains object permanence during periods of occlusion by integrating differentiable rigid body simulation with 3D Gaussian Splatting. Our method separates the scene into individual object Gaussians and corresponding collision meshes. By utilizing differentiable simulation, we estimate parameters such as friction and velocity based on the object’s trajectory prior to occlusion. These estimates generate an SE(3) trajectory that is used to accurately position the object’s Gaussians throughout the occlusion interval.

Since the predicted trajectory adheres to the fundamental equations of rigid body dynamics, it effectively models complex contact events—such as bounces, direction shifts, and friction-induced deceleration—that simple kinematic extrapolation fails to capture. Additionally, we propose a centroid silhouette loss designed to isolate positional gradients from appearance-related noise, resulting in trajectory errors that are 40% lower than those achieved through photometric supervision alone.

We validated our approach using cameras that were excluded from the training set but captured the object during its occlusion. Results from synthetic scene experiments demonstrate that PersistGS surpasses constant velocity extrapolation by 2.46 dB in PSNR and achieves performance within 0.19 dB of the upper bound defined by a ground-truth trajectory.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC