Title: Expressivity of Congruence-Based Architectures for DNNs on Positive-Definite Matrices

Original: arXiv:2606.02490v1 Announce Type: new

Abstract: This study investigates neural network structures designed for the classification of symmetric positive-definite matrices, with a particular emphasis on congruence-like layers. In these layers, the input matrix undergoes multiplication by a weight matrix $W$ (which may be rectangular) and its transpose on the left and right, respectively. This structural element is fundamental to the well-known SPDNet and has also been utilized separately for reducing the dimensionality of positive-definite data. Our findings indicate that the frequent application of (semi)-orthogonality constraints to $W$ restricts the expressive capacity of these layers. Specifically, for certain activation functions, the architecture effectively simplifies to a single-hidden-layer model. This limitation in expressivity arises from a reduction in spectral diversity within congruence-like layers when $W$ is semi-orthogonal, a phenomenon directly attributable to Poincar\'e's separation theorem. Furthermore, we analyze the selection of the final classifier by evaluating various Riemannian classifiers and assessing their interoperability with the feature maps generated by congruence-like layers.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC