Title: Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning

Abstract:

Graph neural networks are currently constrained by two primary limitations stemming from the linear nature of Euclidean vector spaces. First, while existing architectures rely on vectors to encode geometric information—such as directions and gradients—numerous applications necessitate matrix-valued representations to capture inter-directional relationships, for instance, the covariance of atomic orientations within a molecule. Such second-order data is intrinsically modeled by points on the symmetric positive definite (SPD) matrix manifold. Second, conventional message-passing mechanisms employ uniform transformations across edges. Although sheaf neural networks resolve this by utilizing edge-specific transformations, prior implementations have been restricted to vector spaces, preventing the propagation of matrix-valued features.

To overcome these hurdles, we introduce the first sheaf neural network designed to operate natively on the SPD manifold. Our central finding is that the SPD manifold possesses a Lie group structure, which allows for the definition of well-posed sheaf operators without the need to project data back into Euclidean space. From a theoretical standpoint, we demonstrate that SPD-valued sheaves offer strictly greater expressivity than their Euclidean counterparts; specifically, they can represent consistent configurations (global sections) that vector-valued sheaves are incapable of modeling, thereby facilitating more nuanced learned representations. In empirical evaluations, our proposed sheaf convolution successfully converts rank-1 directional inputs into full-rank matrices that encode local geometric structures. Furthermore, our dual-stream architecture sets new state-of-the-art results on six out of seven MoleculeNet benchmarks, with the sheaf framework ensuring robustness across varying model depths.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC