Title: ObjEmbed: Advancing Universal Multimodal Object Embeddings

Abstract:

Bridging the gap between visual elements and their textual descriptions is a core objective and practical necessity in vision-language comprehension. Although contemporary multimodal embedding models have achieved significant success in aligning entire images with corresponding text, they frequently falter when it comes to precise, fine-grained alignment between specific image regions and distinct phrases. To address this, we introduce ObjEmbed, a new embedding framework for Multimodal Large Language Models (MLLMs). This approach breaks down input images into a collection of regional embeddings, with each one representing a distinct object, in addition to global image embeddings. ObjEmbed is capable of supporting various visual understanding applications, including global and local image retrieval, as well as visual grounding.

The model is distinguished by three primary characteristics:

1. Object-Centric Representation: ObjEmbed encapsulates both the spatial and semantic dimensions of objects. For every region, it produces two complementary embeddings: an object embedding designed for semantic matching and an IoU (Intersection over Union) embedding that estimates localization accuracy. The ultimate score for object matching is derived by integrating semantic similarity with the predicted IoU, thereby enhancing retrieval precision.
2. Versatility: The framework effortlessly manages tasks at both the region level and the image level.
3. Encoding Efficiency: It ensures high efficiency by encoding the entire image and all its constituent objects within a single forward pass.

Extensive evaluations across 18 varied benchmarks highlight the model’s superior performance and robust semantic discrimination capabilities.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC