Title: Neural Networks Provably Learn Spectral Representations for Group Composition

Abstract: The emergence of structured internal representations during the training of neural networks is a fundamental question in deep learning research. This study explores this dynamic through the lens of group composition, focusing on a two-layer neural network tasked with predicting the product $g_1 \star g_2$ for elements within a finite group $G$. By transforming the projected gradient flow into the Fourier domain, we show that the training process is driven by a Riemannian gradient ascent acting on an energy functional rooted in representation theory. We establish that, given random initialization, this optimization flow causes individual neurons to almost surely converge toward a single irreducible representation, while cross-layer Fourier coefficients attain a rotational rank-one alignment. This theoretical framework offers a representation-theoretic explanation for feature learning and identifies a new low-rank compression effect for matrix-valued group representations. Furthermore, in the case of Abelian groups, we deliver a comprehensive population-level analysis: random initialization encourages uniform diversification among nontrivial representations and generates Haar-uniform phases. Together, these factors approximate the indicator function through a majority-vote mechanism. Additionally, we demonstrate that both the alignment of phases and the competition between representations occur at exponential convergence rates.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC