Title: QIF Neurons Offer Superior Stability and Performance in Spike-Based Gradient Descent Compared to LIF Neurons

The capacity to effectively train spiking neural networks is a critical prerequisite for both accurate biological modeling and the advancement of neuromorphic computing technologies. Despite their widespread adoption, leaky integrate-and-fire (LIF) neurons suffer from a significant drawback: minute adjustments to parameters can trigger the sudden appearance or disappearance of spikes. This discontinuity disrupts subsequent neural activity, resulting in unstable representations and the permanent silencing of neurons during exact spike-based gradient descent. While recent studies indicate that certain neuron models, including the quadratic integrate-and-fire (QIF) neuron, circumvent these discontinuities to allow for continuous and smooth gradient descent, it has remained uncertain whether these theoretical benefits yield practical improvements.

This study provides empirical evidence that they do, utilizing a controlled comparative analysis of LIF and QIF networks on the widely used Spiking Heidelberg Digits dataset. Our investigation proceeded in two phases. First, we conducted an extensive hyperparameter optimization for both models, which uncovered a distinct performance superiority for QIF neurons. Second, we visualized the associated loss and gradient landscapes. Consistent with the performance metrics, we observed that the loss landscapes for LIF neurons—characterized by their inherent discontinuities—are notably more fragmented, with corresponding gradients exhibiting greater erraticism. Further analysis of individual sample landscapes revealed that these complications stem from alterations in spike temporal ordering, which frequently precipitate disruptive spike events. Consequently, our findings support the substitution of LIF neurons with models featuring continuous spiking dynamics, such as QIF neurons, to enhance gradient descent training.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC