Title: Efficiently Identifying Consistent LLM Responses via Optimal Bayesian Termination

Abstract: A straightforward method for enhancing the precision of Large Language Models (LLMs), particularly in mathematical and reasoning tasks, involves generating multiple responses and selecting the answer that appears most frequently. This study utilizes Bayesian prior knowledge to reduce sampling expenses by terminating the process once a threshold of consistency is met. While calculating the exact posterior distribution is computationally prohibitive, we propose an efficient "L-aggregated" stopping rule that monitors only the counts of the L-1 most common answers. Our theoretical analysis demonstrates that L=3 is sufficient: this simplified approximation guarantees asymptotic optimality, outperforms baseline methods without priors, and enables rapid posterior calculation. In practice, this approach identifies the most consistent LLM response (the mode) with fewer samples, maintaining comparable accuracy while reducing the number of LLM calls—and thus inference costs—by as much as 50%.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC