Title: Graph Set Transformer

Original: arXiv:2606.05116v1 Announce Type: new Abstract: We introduce the Graph Set Transformer (GST), a neural network architecture for learning on sets of graphs, designed for tasks in which per-element predictions depend on set-wide context as well as local structure. Existing architectures, including DeepSets and SetTransformer, require pre-encoded graph embeddings from a separate GNN, creating a bottleneck between feature extraction and set-level contextualisation. In contrast, GST interleaves node-level feature propagation and cross-graph contextual modelling at every layer, fusing the two levels of information through a gating mechanism. We evaluate GST on a controlled synthetic suite designed to isolate set-conditional structural reasoning and on three real-data benchmarks spanning per-atom reaction-centre identification, reaction yield prediction, and image classification. Under matched parameter budgets, GST performs better than the baselines across these settings. An architectural ablation strongly suggests that the interleaving of local and set context contributes substantially to this advantage.

Rewrite: We present the Graph Set Transformer (GST), a novel neural network framework tailored for processing collections of graphs. This architecture is specifically intended for scenarios where predictions for individual elements are influenced by both the broader context of the entire set and the specific local structures of the graphs. Traditional models, such as SetTransformer and DeepSets, rely on pre-computed graph embeddings generated by external Graph Neural Networks (GNNs), which introduces a disconnect or bottleneck between the initial feature extraction phase and the subsequent set-level contextual analysis. GST overcomes this limitation by integrating node-level feature propagation with cross-graph contextual modeling within each layer, combining these distinct information streams via a gating mechanism. To assess its efficacy, we tested GST on a curated synthetic dataset aimed at isolating structural reasoning conditioned on set properties, as well as on three practical benchmarks: identifying reaction centers at the atomic level, predicting reaction yields, and classifying images. When compared against baseline models under equivalent parameter constraints, GST consistently outperforms them. Furthermore, ablation studies of the architecture indicate that the strategic interleaving of local and set-level contexts is a primary driver of this superior performance.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC