Title: Re-examining Neural Processes Through the Lens of Fourier Transform and Volterra Series

Abstract:

A persistent hurdle in both scientific and engineering disciplines is the task of modeling unknown latent functions based on finite sets of irregularly spaced measurements. Neural Processes (NPs), a class of probabilistic functional models, offer a promising avenue for this challenge. Their effectiveness is further enhanced when they incorporate domain-specific symmetries, such as translation equivariance, which boost both generalization capabilities and sample efficiency. However, current translation-equivariant NPs suffer from two primary drawbacks. First, they often rely on stacking generic components with non-linearities, a design choice that hides the nature of the induced function class and reduces interpretability. Second, convolutional approaches typically depend on kernels with local receptive fields and necessitate dense, uniform input grids. While attention-based methods circumvent the grid requirement, they incur a quadratic computational cost relative to the number of observations.

To overcome these limitations, we present two key contributions. Initially, we employ the Volterra expansion to define continuous translation-equivariant operators as sums of higher-order convolutions. This approach provides analytical clarity and allows for efficient approximation using first-order convolutions. Secondly, we develop set Fourier convolutions (SFConvs), a frequency-domain parameterization technique that functions directly on irregularly sampled data points. SFConvs offer approximately global receptive fields and scale linearly with the number of observations. Leveraging these innovations, we introduce two new conditional NPs (CNPs): SFConvCNPs, which combine SFConv blocks with non-linearities, and SFVConvCNPs, which incorporate the Volterra formulation. Our experimental results on both synthetic and real-world datasets confirm the superiority of our proposed methods over existing state-of-the-art baselines.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC