Title: Flow-Transformed Implicit Processes for Function-Space Variational Inference

Original: arXiv:2606.01954v1 Announce Type: new Abstract: Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box {\alpha} objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.

Rewrite: arXiv:2606.01954v1 Announce Type: new Abstract: By employing flexible generative mechanisms to define distributions over functions, implicit-process priors have become a compelling option for Bayesian modeling in function space. Nevertheless, conducting posterior inference with these priors presents significant difficulties, as the resulting function-space distributions rarely possess a closed-form expression. A common pragmatic solution involves approximating the prior via a limited set of sampled functions, subsequently expressing posterior functions as learned linear combinations of these samples. Traditional methods typically assign a Gaussian variational distribution to the combination weights. Although this approach is computationally feasible, it restricts the variety of posterior uncertainty shapes that can be captured, particularly in cases where the true posterior exhibits asymmetry, heavy tails, or multiple modes. To address these limitations, we introduce Flow-Transformed Implicit Processes (FTIP), a variational inference framework designed to enhance the expressiveness of finite-dimensional function-space approximations. Rather than relying on Gaussian distributions for the combination weights, FTIP leverages normalizing flows to construct a more sophisticated variational distribution. This strategy yields a highly flexible posterior over functions without sacrificing optimization tractability. The model is trained via a Black-Box {\alpha} objective, which facilitates a comparison between mass-covering and mode-seeking variational dynamics. Our experimental results demonstrate that FTIP successfully recovers asymmetric and multimodal posterior structures in function space, whereas Gaussian coefficient approximations often result in smoothed or collapsed representations.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC