Title: Establishing Frequentist Consistency in Prior-Data Fitted Networks for Causal Inference

Foundation models utilizing prior-data fitted networks (PFNs) have demonstrated robust empirical results in causal inference by treating the process as an in-context learning challenge. However, the extent to which PFN-based causal estimators deliver uncertainty quantification consistent with classical frequentist methods remains an open question. This study bridges that gap by investigating the frequentist consistency of PFN-based estimators specifically for the average treatment effect (ATE).

First, we demonstrate that current PFNs, when viewed as Bayesian ATE estimators, are susceptible to prior-induced confounding bias. Because the prior is not asymptotically dominated by the data, these models fail to achieve frequentist consistency. To address this limitation, we propose a calibration method grounded in one-step posterior correction (OSPC). Our analysis indicates that OSPC effectively restores frequentist consistency, enabling a semi-parametric Bernstein-von Mises theorem for calibrated PFNs. This implies that both the calibrated PFN-based estimators and traditional semi-parametric efficient estimators converge in distribution as the sample size increases.

Furthermore, we operationalize the OSPC by applying tailored martingale posteriors to PFNs. This approach allows for the recovery of the functional nuisance posteriors necessary for OSPC implementation. In various (semi-)synthetic experimental settings, PFNs calibrated via our martingale posterior OSPC generate ATE uncertainty that (i) aligns asymptotically with frequentist uncertainty and (ii) maintains strong calibration in finite samples, outperforming other Bayesian ATE estimators.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC