Title: GeoSAM-3D: Leveraging Geodesic Prompt Propagation for Open-Vocabulary 3D Scene Segmentation via Monocular Video

Abstract: While conventional open-vocabulary 3D scene segmentation typically relies on RGB-D video streams, calibrated multi-view imagery, or pre-reconstructed meshes, GeoSAM-3D explores a more streamlined approach. In this framework, users simply upload a brief monocular video and identify a target object by clicking or naming it within a single frame; the system then generates a corresponding 3D mask that propagates across a Gaussian scene representation. This solution integrates frozen image and video foundation models with monocular 3D Gaussian Splatting for reconstruction, alongside a differentiable graph-geodesic propagation kernel applied to Gaussian centroids. A pivotal design feature is the use of heat-kernel distance on the reconstructed scene graph for prompt propagation, as opposed to relying on 3D Euclidean nearest neighbors. This method maintains continuity across curved surfaces and effectively minimizes mask leakage between adjacent but distinct objects. The paper outlines the current state of the repository, details the mathematical kernel utilized in geosam3d.propagate, describes the feature head trained using Segment Anything masks, and highlights existing validation mechanisms within the codebase. Furthermore, the evaluation protocol is structured to assess implementation correctness, the efficacy of graph propagation, control over leakage, and interactive latency.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC