Title: Mitigating Optimization Friction in Multimodal Person Re-Identification

Abstract:

The simultaneous optimization of image-to-image (I2I) and text-to-image (T2I) person re-identification (ReID) is frequently impeded by significant modality gaps and contradictory training goals, which ultimately result in inferior shared representations. While I2I ReID prioritizes identity-level invariance across various images of the same individual, T2I ReID relies on instance-specific textual descriptions that correspond to distinct visual characteristics. This study investigates the core distinctions between these two ReID tasks and their respective optimization mechanisms to facilitate more effective training. Because I2I and T2I ReID are typically examined in isolation, the loss functions tailored for one retrieval mode can inadvertently degrade the representation quality essential for the other.

In response to these challenges, we introduce a decoupled, two-stage training framework designed to learn a unified representation across both image and text modalities. This approach utilizes a single vision encoder capable of handling both I2I and T2I retrieval tasks while preventing interference between them during the training phase. We conducted comprehensive experiments involving various configurations, including different domain mixing methods, learning strategies, and task objectives. Our results indicate that pre-training with I2I ReID significantly improves generalization capabilities when applied to T2I data. Furthermore, we discovered that integrating textual supervision during the vision encoder’s training phase boosts performance for both I2I and T2I tasks. We argue that these findings offer a substantial advancement toward the development of unified ReID systems and broader cross-modal retrieval solutions.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC