Title: Optimizing Instance Segmentation via Parameter-Efficient Fine-Tuning of Large Pretrained Models

Abstract

The landscape of artificial intelligence research and application has undergone a significant transformation due to the emergence of large pretrained models, which currently achieve state-of-the-art performance across a wide array of tasks. However, the exponential growth in model parameters has necessitated the development of training strategies that are efficient in terms of parameter usage. Although considerable progress has been made in this area, there remains a scarcity of research focusing on parameter-efficient fine-tuning (PEFT) methods, particularly when applied to transformer-based architectures for instance segmentation.

To bridge this gap, this study evaluates the efficacy of PEFT techniques, specifically focusing on adapters and Low-Rank Adaptation (LoRA). The investigation involves applying these methods to two distinct models across four benchmark datasets. Our approach integrates sequentially arranged adapter modules and introduces LoRA to deformable attention—a technique explored here for the first time. This strategy delivers competitive performance while requiring the fine-tuning of only 1-6% of the model’s parameters, a substantial reduction compared to the 40-55% typically needed for conventional fine-tuning.

Key results suggest that deploying 2-3 adapters per transformer block strikes the ideal balance between efficiency and performance. Additionally, LoRA demonstrates high parameter efficiency when utilized with deformable attention, occasionally outperforming adapter-based configurations. These findings highlight that the effectiveness of PEFT techniques is contingent upon both dataset complexity and model architecture, emphasizing the necessity for context-specific tuning. Ultimately, this work illustrates the potential of PEFT to facilitate scalable, customizable, and computationally efficient transfer learning for instance segmentation.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC