Title: ShelfAware: Enabling Real-Time Semantic Localization in Quasi-Static Settings Using Affordance Sensors

Abstract: Standard vision-based localization methods often struggle in indoor workspaces that are quasi-static, where the global geometric structure remains constant while local semantics evolve continuously. This dynamic nature creates repetitive geometry, moving clutter, and perceptual noise that typically hinder accurate positioning. To address these challenges, we introduce ShelfAware, a semantic particle filter designed for robust global localization. Unlike traditional approaches that rely on fixed landmarks, ShelfAware interprets scene semantics as statistical evidence across object categories. By combining depth likelihood with category-centric semantic similarity, the system utilizes a precomputed repository of semantic viewpoints to generate inverse semantic proposals within Monte Carlo Localization (MCL). This approach enables rapid, targeted hypothesis generation on inexpensive, vision-only hardware.

To demonstrate the system’s scalability independent of specific perception models, we evaluated ShelfAware across two distinct domains. In a strictly controlled mock retail setting, the system achieved a 97% success rate in global localization and maintained a top-tier tracking success rate of 66% across various conditions, including cart-mounted, wearable, and dynamic occlusion scenarios. Additionally, in a 3,500-square-foot operational grocery store using an open-vocabulary vision pipeline, ShelfAware significantly surpassed both geometric and fixed-quantity semantic baselines. By treating semantics as a distribution and employing inverse proposals, ShelfAware effectively resolves geometric aliasing, offering a robust, infrastructure-free solution for mobile and assistive robots operating in dynamic real-world environments.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC