Title: Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference

Abstract: As Large Language Models (LLMs) become deeply embedded in high-performance computing (HPC) pipelines, they are driving scientific breakthroughs across various domains, including code generation and specialized decision-making. However, the mechanisms by which soft errors propagate and impact LLM inference processes remain poorly understood. To address this knowledge gap, this paper presents a thorough investigation into error propagation within LLM inference, facilitated by LLMFI—a novel, deterministic, and configurable fault-injection framework. Leveraging LLMFI, we systematically introduce faults into three open-weighted LLMs across thirteen representative tasks spanning reasoning, multilingual, mathematical, and coding challenges. Furthermore, our detailed case studies uncover specific vulnerability patterns. The study results in 17 key insights that deepen the comprehension of error propagation in LLM inference and propose four low-overhead, software-only strategies to enhance reliability, providing actionable guidance for future error detection and mitigation efforts.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC