Title: The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation

Abstract: Large language models (LLMs) are increasingly tasked with generating significant volumes of production-grade code, frequently for problems that admit multiple valid algorithmic approaches. Research indicates that incidental prompt cues—such as contextual words or metadata not directly specified in the task requirements—can influence the specific algorithm a model selects, even when all resulting outputs satisfy identical testing criteria. While prompt sensitivity is widely recognized as a method to enhance output quality, this study focuses on "output policy," defined here as the selection of algorithms under conditions of fixed correctness. We conceptualize "algorithm steering" as cue-driven changes in the distribution of algorithm families. To investigate this phenomenon, we conducted 46,535 controlled experiments spanning 11 distinct tasks, 19 types of cues (including 18 channels and a semantic-vs-surface ablation of memoization that alters typography and punctuation while preserving meaning), and 15 model configurations. Our findings reveal substantial, systematic shifts in algorithm-family distributions, with variations reaching up to 100 percentage points. These shifts generally align with the semantics of the cues and persist even in applied scenarios such as rate limiting. Among the mitigation strategies tested, explicitly naming the algorithm proved to be the most effective. Consequently, accidental contextual information creates an "invisible lottery" affecting performance, security, and code maintainability.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC