Title: Enhancing the Trustworthiness of Long-Form Text Creation Through Hallucination Rejection Sampling

Abstract:

While Large Language Models (LLMs) have made significant strides in generating open-ended text, their susceptibility to producing inaccurate or unsupported information remains a critical barrier to reliability. This problem is particularly acute in long-form generation, where a phenomenon known as "hallucination snowballing" causes initial errors to cascade and amplify throughout the output. To counteract this, we introduce Segment-wise HAllucination Rejection Sampling (SHARS), a novel framework designed to mitigate hallucinations during the inference phase. SHARS employs a flexible hallucination detector to identify and discard erroneous segments as they are generated, continuing to resample until the content is factually faithful. By ensuring that subsequent generation steps are built exclusively upon verified, high-confidence information, the framework effectively curbs the accumulation of hallucinations and boosts factual consistency.

To implement this approach, we utilize semantic uncertainty as the detection mechanism, incorporating several key adjustments to overcome its inherent limitations and optimize its performance for long-form texts. A significant advantage of our method is that it allows models to self-correct hallucinations without relying on external tools like web search engines or knowledge bases, although it remains compatible with such resources for potential future enhancements. Experimental results on standard hallucination benchmarks indicate that our technique significantly lowers hallucination rates in long-form outputs while maintaining, or even enhancing, the informational value of the text. The source code for this work is publicly accessible at: https://github.com/TreeLLi/hallucination-rejection-sampling.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC