Title: Take a Peek: Efficient Encoder Adaptation for Few-Shot Semantic Segmentation via LoRA

Abstract

Few-shot semantic segmentation (FSS) is designed to identify and segment novel object classes within query images, relying solely on a limited set of annotated examples. While previous studies have predominantly concentrated on optimizing decoders, the encoder’s constrained capacity to extract pertinent features for unseen categories continues to serve as a primary bottleneck. To address this, we present Take a Peek (TaP), a straightforward yet powerful approach that boosts encoder adaptability for both standard and cross-domain FSS tasks. TaP achieves this by inducing a lightweight feature-space shift that is conditioned on the support set.

Utilizing Low-Rank Adaptation, our method fine-tunes the encoder based on the support set, incurring minimal computational costs. This allows for rapid adaptation to new classes while effectively reducing the risk of catastrophic forgetting. Being model-agnostic, TaP can be effortlessly incorporated into existing FSS frameworks. We conducted extensive evaluations across various benchmarks, including COCO $20^i$, Pascal $5^i$, and cross-domain datasets such as DeepGlobe, ISIC, and Chest X-ray. The results indicate that TaP consistently enhances segmentation accuracy across different models and shot configurations.

Notably, TaP yields substantial improvements in complex multi-class environments, underscoring its utility in real-world applications. Furthermore, an analysis of rank sensitivity reveals that high performance is attainable even with low-rank adaptations, thus guaranteeing computational efficiency. By tackling a critical weakness in FSS—the encoder’s ability to generalize to novel classes—TaP advances the development of more robust, efficient, and generalizable segmentation systems. The source code is publicly accessible at https://github.com/pasqualedem/TakeAPeek.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC