Title: Stabilizing Weakly Supervised Incremental Segmentation Through Semantic Anchors and Spatial Arbitration

Original: arXiv:2606.04060v1 Announce Type: new Abstract: Weakly Incremental Learning for Semantic Segmentation (WILSS) suffers from the continuous introduction of noisy supervision, which progressively corrupts class-level representations, leading to severe feature drift and semantic corruption, thereby causing newly learned classes to overwrite old ones. To address these issues, we propose a drift-resilient WILSS approach, named SASA, designed to stabilize semantic learning via Semantic Anchors and Spatial Arbitration. Specifically, at the representation level, we introduce semantic anchors of learnable tokens as rigid class-level references to preserve long-term semantic identity. Complementary to this, an elastic residual adaptation facilitates controlled, instance-specific refinement, ensuring a stable yet flexible learning trajectory. At the supervision level, we develop a Spatial Label Arbitration mechanism that performs geometry-aware decisions to directly filter unreliable signals and enforce a strict "one object, one class" constraint. By synergistically stabilizing representations and improving supervision reliability, SASA effectively mitigates feature drift under weak supervision. Extensive experiments on standard benchmarks demonstrate that our approach consistently outperforms existing state-of-the-art methods, particularly in challenging multi-step incremental settings. The code is available at https://github.com/ZhonggaiWang/SASA.

Rewrite: Weakly Incremental Learning for Semantic Segmentation (WILSS) is hampered by the persistent influx of noisy supervision, which gradually degrades class-level representations. This degradation results in significant feature drift and semantic corruption, ultimately causing newly acquired classes to overwrite previously learned ones. To counteract these challenges, we present SASA, a drift-resilient WILSS framework that stabilizes semantic learning through the integration of Semantic Anchors and Spatial Arbitration.

At the representation level, SASA employs semantic anchors composed of learnable tokens. These act as fixed, class-level references to maintain long-term semantic integrity. In conjunction with this, an elastic residual adaptation mechanism enables controlled, instance-specific refinements, thereby ensuring a learning path that is both stable and adaptable.

On the supervision front, we introduce a Spatial Label Arbitration mechanism. This component utilizes geometry-aware decision-making to directly eliminate unreliable signals and strictly enforce the "one object, one class" rule. By simultaneously stabilizing feature representations and enhancing the reliability of supervision, SASA successfully reduces feature drift in weakly supervised environments. Comprehensive experiments on established benchmarks show that our method consistently surpasses current state-of-the-art techniques, with notable improvements in difficult multi-step incremental scenarios. The source code can be accessed at https://github.com/ZhonggaiWang/SASA.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC