Title: A Spectral Examination of In-Context Operator Networks

Abstract:

Current assessments of in-context operator learning and neural operators predominantly depend on prediction error metrics. However, achieving precise output predictions does not necessarily ensure the preservation of the correct local dynamical structure. A model might successfully match solution trajectories while simultaneously displaying flawed sensitivities, altered frequency responses, spurious mode coupling, or unstable tangent dynamics. To address this, we propose a Jacobian-based spectral audit tailored for in-context operator learning. By fixing a prompt and differentiating the network’s output with respect to the query function, we treat the resulting Jacobian as a learned tangent operator. We then project this operator onto Fourier modes to derive a local spectral characterization of the inferred operator, capturing details such as phase structure, frequency-dependent gains, and cross-mode interactions.

This auditing method supplements conventional prediction metrics by verifying whether the model accurately reproduces the local mechanisms of the underlying PDE operator, rather than merely fitting the outputs. Our analysis across various benchmarks uncovers distinct operator-level behaviors, including phase transport, nonlinear mode coupling, reaction-diffusion stability structures, and damping that varies with viscosity. Furthermore, the audit exposes failures that standard prediction-error metrics often obscure, such as high-frequency degradation, inaccurate phase recovery, and inconsistencies between prompts and operators. Notably, we find that corrupted or internally inconsistent prompts can degrade the tangent-operator structure even when pointwise predictions retain partial accuracy. These findings indicate that prediction accuracy and local operator fidelity are separate attributes of learned neural operators. Consequently, our framework serves as a valuable diagnostic tool for evaluating stability, sensitivity, and operator consistency.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC