Title: Enhancing Neural Algorithmic Reasoning with Auxiliary Reconstruction for Richer Representations

Abstract:

Neural algorithmic reasoning has become a prominent area of study, focusing on training neural networks to emulate the sequential logic of traditional rule-based algorithms. In this context, algorithm execution is conceptualized as a series of states, with each state capturing the intermediate result following a specific step. The primary goal is to train models to produce state sequences that accurately mirror the underlying algorithmic workflow. A prevalent approach utilizes an encoder-processor-decoder framework: the encoder generates state representations, the processor simulates the algorithmic transitions, and the decoder reconstructs the output states. Although previous research has largely concentrated on optimizing the processor component, the encoder’s contribution to representation learning has been largely overlooked. Most existing solutions employ basic Multi-Layer Perceptron (MLP) encoders, prompting questions about whether these representations possess sufficient detail to support complex algorithmic reasoning.

This study explores methods to elevate encoder representations for neural algorithmic reasoning. We introduce a reconstruction module designed to retrieve the original input state from its encoded form. By adding this auxiliary reconstruction task to the training process, we encourage the encoder to preserve vital information about the input data. Our findings indicate that integrating this task boosts the performance of current neural architectures on standard benchmarks. Additionally, we note that existing encoders frequently fail to fully exploit the correlations between features within a single state. To mitigate this, we leverage insights from self-supervised learning to create an advanced version of the auxiliary task, which motivates the encoder to identify intra-state feature dependencies. Experimental evidence confirms that our approach allows encoders to learn more robust representations, ultimately improving the effectiveness of existing processors in algorithmic reasoning tasks.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC