Title: Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structures

Original: arXiv:2606.00557v1 Announce Type: new

Abstract:

Gaining a comprehensive understanding of how neural networks (NNs) operate and generate predictions requires looking beyond the input domain; it is now widely recognized that inspecting the internal inference mechanisms is crucial to capturing the full picture. To elucidate these internal processes, it is necessary to evaluate the significance of latent representations relative to specific tasks. This study introduces the normalized relevance measure (NRM) framework, a novel, general-purpose explanation method that assigns relevance scores to any set of neurons within any layer of any architecture. Within this framework, the relevance of chosen neurons is rigorously defined as a normalized signed measure. This measure is derived through straightforward marginalization and conditioning operations, adhering to additive and multiplicative laws, much like standard probability measures. Crucially, the normalization feature ensures that relevance scores remain comparable across different layers. By explicitly identifying the quantities computed by existing propagation-based explanation algorithms, the NRM framework encompasses these prior methods. We illustrate the framework’s effectiveness in computer vision tasks, where joint relevance analysis across multiple layers uncovers critical information flows within VGG16 networks. Ultimately, the NRM framework offers a robust, mathematically sound approach for deciphering information propagation in modern neural networks, establishing a versatile and widely applicable foundation for explainable artificial intelligence.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC