Title: Seg-Zero: Enhancing Segmentation Through Cognitive Reinforcement and Reasoning Chains

Abstract:

Conventional approaches to reasoning-based segmentation typically depend on supervised fine-tuning utilizing categorical labels and basic descriptions. This reliance restricts the model’s ability to generalize outside its training domain and fails to provide transparent reasoning pathways. To overcome these challenges, we introduce Seg-Zero, an innovative framework that leverages cognitive reinforcement to establish explicit chain-of-thought reasoning while demonstrating exceptional generalizability.

Seg-Zero employs a decoupled architecture comprising two distinct components: a reasoning model and a segmentation model. The reasoning module is tasked with interpreting user intent, generating clear reasoning chains, and formulating positional prompts. These prompts then guide the segmentation model in producing precise pixel-level masks. To steer the optimization process effectively, we have developed a sophisticated reward mechanism that combines both format and accuracy rewards.

Notably, Seg-Zero is trained solely through reinforcement learning using GRPO, operating without the need for explicit reasoning data. This approach enables the model to achieve robust zero-shot generalization and reveals emergent reasoning capabilities during testing. Experimental results indicate that Seg-Zero-7B secures a zero-shot score of 57.5 on the ReasonSeg benchmark, outperforming the previous state-of-the-art LISA-7B by 18%. This substantial gain underscores Seg-Zero’s capacity to generalize across diverse domains while simultaneously offering a clear, explicit reasoning process.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC