Title: InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation

Abstract:

Large language models (LLMs) that demonstrate safety during testing phases may still generate dangerous content when deployed in real-world scenarios. Due to the inherent stochasticity of sampling, identical prompts can yield varied outputs, meaning that low-probability harmful responses can still reach users at scale. Traditional human evaluation methods typically involve generating numerous random samples per prompt and reviewing them in static spreadsheets. This approach lacks scalability, as it forces reviewers to repeatedly process nearly identical text prefixes. To overcome these limitations, we introduce InFerActive, an interactive platform that represents sampling outcomes as an explorable tree structure composed of readable phrases. This interface enables evaluators to dynamically filter, navigate, and expand the generation space. InFerActive employs a novel tree construction technique known as breadth-first sampling, which achieves harmful-response coverage comparable to random sampling while reducing the number of required samples by as much as 5.0 times. Results from two controlled user studies (N = 12 per study) indicate that InFerActive substantially enhances both evaluation efficiency and coverage when compared to conventional spreadsheet methods and basic tree-based baselines.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC