Title: In-Context Graphical Inference

Original: arXiv:2606.05042v1 Announce Type: new Abstract: Marginal inference in discrete graphical models forces a choice between exactness and scalability: exact algorithms are intractable for high-treewidth graphs, while iterative approximations (Belief Propagation, variational methods) sacrifice convergence guarantees on frustrated topologies. We argue that this dichotomy stems from a mismatched inductive bias: iterative methods abandon the sequential elimination structure that makes exact inference correct. We introduce In-Context Graphical Inference (ICG-I), an autoregressive Graph Transformer that restores this structure by mimicking Variable Elimination with learned, Tensor- Train-compressed intermediate factors, paired with a Dirichlet output layer and Weighted Conformal Prediction for calibrated, distribution-free coverage guarantees under topological shift. We prove that TT compression errors propagate at most lincarly through the autoregressive chain, that the Dirichlet-Multinomial loss is a proper scoring rule, and that WCP maintains coverage with a quantifiable degradation under estimated density ratios. We conducted intensive experiments to evaluate ICG-I and achieved state-of-the-art performance across all benchmarks. ICG-I reduces MAE from 0.041 (best baseline) to 0.020 on standard instances and achieves 0.048 on N=500 frustrated spin glasses where BP diverges entirely.

Rewrite: arXiv:2606.05042v1 Announcement Type: new Abstract: When performing marginal inference on discrete graphical models, practitioners face a trade-off between precision and computational feasibility. While exact methods become unmanageable for graphs with high treewidth, iterative approximation techniques—such as Belief Propagation and variational approaches—often fail to converge reliably on frustrated network structures. We posit that this fundamental conflict arises from an incompatible inductive bias: iterative strategies discard the sequential elimination framework that ensures the correctness of exact inference. To address this, we present In-Context Graphical Inference (ICG-I), an autoregressive Graph Transformer designed to reintroduce this structural integrity. ICG-I emulates Variable Elimination by utilizing learned, Tensor-Train-compressed intermediate factors. The model incorporates a Dirichlet output layer alongside Weighted Conformal Prediction (WCP) to provide calibrated, distribution-free coverage assurances even when topological conditions shift. Our theoretical analysis demonstrates that errors from Tensor-Train compression increase at most linearly throughout the autoregressive sequence, confirms that the Dirichlet-Multinomial loss functions as a proper scoring rule, and shows that WCP preserves coverage levels with only measurable decline under estimated density ratios. Extensive experimental evaluations confirm that ICG-I delivers state-of-the-art results across every benchmark tested. Specifically, the method lowers the Mean Absolute Error (MAE) from 0.041, the best achieved by existing baselines, down to 0.020 on standard test cases. Furthermore, on N=500 frustrated spin glass instances where Belief Propagation completely fails to converge, ICG-I achieves an MAE of 0.048.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC