Title: The Architecture of Summation: Unveiling the Geometric Underpinnings of Arithmetic within Large Language Models

Original: arXiv:2606.03645v1 Announce Type: cross Abstract: Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual stream geometry during multi-operand addition, we identify the Iso-Raw-Sum Trajectory (IRST), a geometric structure where representations are anchored by semantic digits and modulated by continuous carry fibers. We propose the Noisy Quantization Model to explain this geometry, framing arithmetic errors as Geometric Slippages caused by internal neural noise pushing a continuous, latent Carry Potential across quantization thresholds. This geometric framework further elucidates Probe Versatility, explaining how lightweight probes can disentangle coexisting latent signals (such as ground truth versus hallucination) from a single activation vector. Finally, we validate these insights through a geometric consistency check method that effectively detects and corrects these quantization failures during inference. Our code is available at https://github.com/RL-MIND/Shape-of-Addition.

Rewrite: Large Language Models display a contradictory weakness in basic arithmetic, suggesting a misalignment between their internal processing mechanisms and their final discrete outputs. Through an examination of the residual stream’s geometry during additions involving multiple operands, this study reveals the Iso-Raw-Sum Trajectory (IRST). This specific geometric configuration features representations fixed by semantic digits and influenced by continuous carry fibers. To account for this structure, we introduce the Noisy Quantization Model, which interprets arithmetic mistakes as "Geometric Slippages." These slippages occur when internal neural noise drives a continuous, latent Carry Potential across quantization boundaries. Furthermore, this geometric perspective sheds light on Probe Versatility, demonstrating how simple probes can separate overlapping latent signals—such as distinguishing factual data from hallucinations—within a single activation vector. We confirm these findings using a geometric consistency check technique capable of identifying and rectifying these quantization errors during the inference process. The source code for this research can be accessed at https://github.com/RL-MIND/Shape-of-Addition.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC