Title: Compressing Neural Networks via Approximate Differential Equivalence

Abstract:

While magnitude-based pruning is the standard technique for reducing neural network size by eliminating parameters with low local importance, we introduce a complementary strategy that focuses on aggregating neurons exhibiting similar functional behaviors rather than deleting weights in isolation. In this approach, a trained network is first represented as a polynomial system of ordinary differential equations (ODEs). We then employ a lumping technique known as Approximate Forward Differential Equivalence to group neurons that demonstrate nearly identical induced dynamics. The degree of compression is governed by a single tolerance parameter, $\varepsilon$, which facilitates a smooth balance between the model’s compactness and its predictive performance.

We tested this methodology on two fronts: synthetic datasets generated from nonlinear dynamical systems with known ground-truth behaviors, and standard public regression benchmarks. In both scenarios, our method delivered significant reductions in parameters without compromising accuracy. Furthermore, it consistently outperformed or matched magnitude-based pruning and the Wanda algorithm at comparable compression rates. These findings indicate that leveraging differential equivalence for aggregation offers a robust and principled alternative to traditional weight-centric pruning methods.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC