Title: Assessing the Resilience of Visual State Space Models

Abstract:

Vision State Space Models (VSSMs) represent a novel architectural approach that merges the capabilities of recurrent neural networks with latent variable models. These models have achieved impressive results in visual perception by efficiently managing long-range dependencies and modeling intricate visual dynamics. Despite this success, their robustness against both natural and adversarial disturbances remains a significant point of concern.

This study provides a thorough evaluation of VSSM robustness across a variety of perturbation conditions, such as image structure variations, occlusions, common corruptions, and adversarial attacks. We benchmark their performance against established architectures, including Convolutional Neural Networks (CNNs) and transformers. Additionally, we examine how VSSMs handle object-background compositional shifts using advanced benchmarks designed to gauge performance in complex visual environments. The study also evaluates robustness in object detection and segmentation tasks by utilizing corrupted datasets that simulate real-world conditions.

To further elucidate the adversarial robustness of VSSMs, we perform a frequency-based analysis of adversarial attacks, testing their effectiveness against both low- and high-frequency perturbations. Our results delineate the strengths and weaknesses of VSSMs when dealing with complex visual corruptions, providing critical insights for subsequent research. The associated code and models will be accessible at https://github.com/HashmatShadab/MambaRobustness.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC