Title: Optimizing Reliability in Long-Form Text: The Case for Selective Abstraction in LLMs

Abstract:

Despite their widespread deployment, Large Language Models (LLMs) are susceptible to factual inaccuracies, a vulnerability that undermines user confidence and hinders their integration into high-stakes environments. A common strategy to address this issue involves integrating uncertainty estimation mechanisms that allow models to withhold responses when their confidence levels are insufficient. However, this binary "all-or-nothing" method is often too rigid for long-form generation, frequently resulting in the unnecessary loss of valuable content.

To address these limitations, we present Selective Abstraction (SA), a novel framework that allows LLMs to balance specificity against reliability. By selectively diminishing the detail of uncertain information, SA enables models to maintain accuracy without sacrificing coherence. We define this framework formally using the concepts of selective risk and coverage. Specifically, we introduce Atom-wise Selective Abstraction, a claim-level implementation that breaks down model outputs into atomic claims—concise, self-contained statements representing single facts. When uncertainty is detected, these atoms are substituted with broader, higher-confidence abstractions.

To assess the efficacy of this approach, we constructed a new end-to-end pipeline for open-ended generation. Within this pipeline, risk is quantified by factual correctness, while coverage is evaluated through an information-theoretic metric that tracks retained information. Our experiments across six open-source models on the FactScore and LongFact-Objects benchmarks reveal that Atom-wise SA consistently surpasses existing baseline methods. Notably, it enhances the area under the risk-coverage curve (AURC) by as much as 27.73% compared to simple claim removal. These findings demonstrate that lowering specificity can significantly improve model accuracy and reliability while preserving the core meaning of the generated text.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC