Title: KG-Guard: A Graph-Based Approach to Detecting Hallucinations in Knowledge Base Question Answering

Abstract:

Large language models (LLMs) are seeing growing adoption in knowledge base question answering (KBQA), a task that involves retrieving answers by selecting entities from a subgraph specific to the query. However, LLMs are prone to hallucinations across various applications, and KBQA is no exception. Even when supplied with a graph as the primary knowledge source, models may default to parametric knowledge rather than utilizing the provided graph evidence, or they may execute invalid reasoning over the existing relations. These hallucinated answer nodes pose significant barriers to the practical implementation of KBQA systems, particularly in critical sectors like healthcare.

To address this, we frame hallucination detection in KBQA as a problem of classifying answer nodes. We introduce a lightweight, graph-based framework that operates on the LLM used for answering as a black box. Our method, \methodname, models each KBQA instance as an augmented graph. This process involves initializing node features with semantic representations of knowledge graph (KG) entities, assigning learned vectors to topic entities and the answer nodes proposed by the LLM, and linking a virtual question node to the topic entities. A graph encoder subsequently generates verification-focused node representations. Finally, a small multi-layer perceptron (MLP) classifies each proposed answer node by combining its graph representation with the question embedding.

Evaluations on the WebQSP, ComplexWebQuestions, and PUGG benchmarks demonstrate that our detector attains the highest F1 scores across all three datasets (82.0, 87.4, and 84.3, respectively). This performance surpasses baselines based on LLM-as-a-judge and sampling methods, all while utilizing approximately 305 times fewer parameters than the reference approaches. Furthermore, the feedback provided at the node level is highly actionable. When answers flagged by our detector are reintroduced to the KBQA system for iterative refinement, downstream KBQA F1 scores increase by 13.0–14.5 points, and Exact Match scores improve by 16.9–17.6 points.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC