Title: Linear Probes Identify Task Format, Not Reasoning Mode in Language Model Hidden States

Abstract

Linear probing of hidden states in large language models (LLMs) has become a standard method for asserting that these systems develop distinct internal representations for various types of reasoning. To evaluate this claim, we applied linear probes to the Qwen3-14B model across three benchmarks that represent the classical reasoning trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $\alpha$NLI (abductive).

At layer 32 out of 40, our linear probes achieved a cross-validated accuracy of 100%, exhibiting well-separated geometric structures within the hidden states. Specifically, the intrinsic dimensionalities were measured at 20.6, 28.5, and 33.6 for the respective tasks, with convex hull contamination remaining at or below 1.5%. However, our analysis reveals that this clear separation is entirely attributable to format confounds. When we residualized for source identity, the number of options, and response length, the probe accuracy dropped to chance levels.

Further investigation using trace-anchor similarity showed substantial shared reasoning across tasks, with an agreement rate of 42.5% compared to a 33.3% chance baseline. Additionally, causal steering experiments involving random controls ($n=20$) demonstrated no functional connection between the observed geometric structures and the reasoning mode ($p=0.286$). These findings suggest that high probe accuracy primarily reflects task format rather than underlying computational structures. This highlights the necessity of routinely deconfounding for format in mechanistic interpretability studies.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC