Title: Investigating How LLMs Resolve Conflicts Between Pre-Evidence Beliefs and Retrieved Context in RAG Fact-Checking

Abstract: In the realm of Retrieval-Augmented Generation (RAG) for fact-checking, Large Language Models (LLMs) are frequently employed as verification agents to validate claims against retrieved evidence. However, the parametric knowledge embedded within these models can create pre-evidence biases that may contradict the retrieved context. Current evaluation frameworks fail to adequately characterize this discrepancy between prior beliefs and contextual data, nor do they assess how verifiers manage the tension between parametric and contextual signals. To address this gap, we present \textsc{PAVE} (\emph{Prior-Aware Verifier Evaluation}), a diagnostic testbed designed to stratify LLM verifiers into four distinct epistemic states defined by the accuracy and confidence of their pre-evidence priors. This benchmark evaluates arbitration behavior by determining whether models cling to correct priors despite misleading evidence or successfully correct erroneous priors when presented with accurate evidence. Our experiments involving seven different LLMs demonstrate that prior-context arbitration is inconsistent and highly dependent on the specific model, underscoring the critical need for careful verifier selection in practical RAG-based fact-checking systems. Leveraging these insights, we introduce a lightweight, JSD-based method for test-time arbitration that enhances factual reliability without requiring modifications to the underlying model architecture, delivering competitive performance across various LLM families.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC