Title: RelWitness: Open-Vocabulary 3D Scene Graph Generation with Visual-Geometric Relation Witnesses

Abstract:

The generation of open-vocabulary 3D scene graphs aims to capture object instances and their interrelations using flexible natural-language predicates. However, the primary challenge extends beyond simple vocabulary expansion to include the reliability of supervision. In existing 3D scene graph datasets, relation annotations are often selective, leaving many valid object-pair connections unmarked. To address this, we introduce RelWitness, a framework designed for open-vocabulary 3D scene graph generation from posed RGB-D sequences, even when relation supervision is incomplete.

The core innovation of RelWitness is the "relation witness," defined as a specific visual-geometric cue that renders a relationship observable within the captured scene. For instance, support relations necessitate physical contact and vertical ordering; containment requires enclosure; proximity demands metric closeness; orientation relies on facing direction; and stable relations must endure across different viewpoints where both objects remain visible.

RelWitness generates relation witness records by synthesizing data from RGB views, depth maps, reconstructed 3D geometry, role-sensitive text, object-prior null views, and multi-view consistency checks. A visual-geometric witness verifier then categorizes unannotated relation candidates into three groups: verified missing positives, reliable negatives, or uncertain unlabeled instances. Subsequently, a witness-guided positive-unlabeled learning objective enables the model to learn from incomplete annotations, preventing the erroneous classification of every missing label as a negative.

Additionally, the study presents a witness-consistent decoding method and an RGB-D missing-relation audit protocol. Simulated manuscript-planning experiments conducted on 3DSSG/3RScan and ScanNet-derived open-vocabulary splits demonstrate the framework's intended efficacy, including enhanced recognition of unseen relations, increased witness precision, decreased hallucination, and a reduction in redundant relation phrases. Please note that all numerical results presented are planning values and must be substituted with reproduced measurements prior to submission.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC