Title: REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

Abstract:

While large language models (LLMs) demonstrate robust capabilities across a wide array of tasks, they are still susceptible to hallucinations. This vulnerability underscores the necessity of rigorously assessing their reliability when exposed to realistic adversarial inputs. In this work, we define the challenge of eliciting hallucinations as a constrained optimization problem, aiming to identify adversarial prompts that maintain semantic coherence and equivalence to standard, benign user queries.

Current attack methodologies exhibit significant constraints. Discrete, prompt-based approaches ensure semantic validity and coherence but are restricted to a narrow range of prompt variations. Conversely, continuous latent-space attacks navigate a more expansive search space but frequently result in decoded prompts that lose their validity as accurate rephrasings. To overcome these drawbacks, we introduce REALISTA, a novel framework for realistic latent-space attacks.

REALISTA generates an input-specific dictionary of valid editing directions, where each direction represents a semantically equivalent and coherent rephrasing. The framework then optimizes continuous combinations of these directions within the latent space. This architecture effectively merges the optimization versatility of continuous methods with the semantic authenticity of discrete, rephrasing-based strategies.

Our experimental results show that REALISTA delivers performance that is either superior to or on par with current state-of-the-art realistic attacks on open-source LLMs. Most notably, it successfully compromises large reasoning models in free-form response scenarios, a context where previous realistic attacks have proven ineffective. The code for this project is accessible at https://github.com/Buyun-Liang/REALISTA.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC