Title: EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers

Abstract

Visual explainability for object detection presents challenges associated with the multi-instance nature of detection tasks. Current methods primarily utilize post-hoc paradigms, including gradient-based or perturbation-based explanation techniques, to interpret pretrained detectors. These approaches necessitate additional gradient computations or repeated model inferences, which can limit computational efficiency.

This study introduces the End-to-end Instance-specific Visual Explanation framework (EIVE), designed to generate instance-level saliency maps directly following the forward pass of Detection Transformer (DETR)-like models. The framework reformulates the cross-attention mechanism within the decoder as an instance-level feature attribution pathway, wherein the cross-attention of each object query corresponds to the visual attribution of its predicted instance. A cross-layer hybrid consensus fusion (CLHCF) module is implemented to aggregate cross-attention signals across decoder layers, resulting in stable and compact explanations.

The EIVE explanation process eliminates the need for gradient computation or input perturbation, providing high computational efficiency. The framework is applicable to both single- and multi-scale DETR-like object detectors. Additionally, an attention-aware joint training strategy (AAJTS) is proposed as a training-oriented application. This strategy imposes spatial constraints on cross-attention patterns to encourage stable and concentrated attribution representations.

Experiments conducted on the MS COCO 2017, ExDark, and Cityscapes datasets indicate that EIVE generates high-quality instance-level saliency maps. The method achieves performance comparable to or exceeding state-of-the-art post-hoc methods across standard metrics, while demonstrating significant improvements in explanation efficiency. Code is available at https://github.com/xjlDestiny/EIVE.git.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC