Title: Leveraging Fusion-Time Spectral Reliability for Adaptive Fusion and Expert Routing in RGB-Infrared Object Detection

Abstract:

Standard RGB-infrared detection frameworks often overlook the statistical information generated during cross-modal fusion, resulting in downstream components lacking insight into the reliability of the current interaction. To address this, we introduce a parameter-free, seven-dimensional spectral reliability descriptor that encapsulates key metrics such as band energy, amplitude ratios, phase consistency, and cross-modal correlation. Rather than discarding this information after fusion, we repurpose the descriptor to enhance two distinct processes: Spectral Reliability Fusion (SRF), which modulates spectral residuals against a robust spatial baseline, and Reliability-Conditioned Expert Routing (RCER), which merges the descriptor with pooled content to guide sparse, post-fusion experts.

Our ablation studies demonstrate that gating mechanisms informed by the descriptor outperform those relying solely on content for mAP50 metrics. Furthermore, a $2{\times}2$ factorial analysis reveals that routing conditioned on the descriptor yields a more significant marginal improvement over the expert architecture itself, achieving this with a comparable parameter count. When evaluated under six synthetic degradation scenarios on the DroneVehicle dataset, our approach achieves an average retention rate of 95.0%, surpassing both content-only Mixture of Experts (92.0%) and simple concatenation (87.9%). Notably, the most substantial gains occur under modality dropout conditions. Additionally, the model boosts mAP50 by +5.2 and +5.3 on natural day and night splits, respectively. These findings indicate that maintaining fusion-time reliability as a distinct signal significantly enhances both adaptive fusion mechanisms and post-fusion conditional computation.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC