Title: Reasoning Shift: How Context Silently Shortens LLM Reasoning

Abstract:

While large language models (LLMs) that exhibit test-time scaling behaviors—such as extended reasoning traces and self-verification—have achieved impressive results on complex, long-horizon reasoning tasks, the robustness of these capabilities has not been thoroughly examined. To address this gap, we perform a systematic evaluation of several reasoning models across three distinct scenarios: problems embedded within lengthy, irrelevant context; multi-turn conversations involving independent tasks; and problems framed as subtasks within larger, complex objectives.

Our analysis uncovers a notable trend: when the same problem is presented under varying contextual conditions, reasoning models generate significantly shorter reasoning traces—reductions of up to 65%—compared to when the problem is isolated. A more granular investigation indicates that this compression correlates with a decline in self-verification and uncertainty management practices, such as double-checking work. Although this shift in behavior does not impact performance on simpler tasks, it may hinder effectiveness on more difficult challenges. Furthermore, we demonstrate that targeted supervised fine-tuning can partially alleviate the negative impacts of irrelevant context. We aim for these insights to highlight the need for greater attention to the robustness of reasoning models and the critical issue of context management in LLMs and LLM-based agents.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC