Title: CA-BED: Conversation-Aware Bayesian Experimental Design

Abstract:

While Large Language Models (LLMs) demonstrate strong capabilities in static reasoning tasks, their effectiveness frequently diminishes in interactive contexts that require the active gathering of information via questioning. A primary obstacle in these scenarios is the selection of inquiries that effectively reduce uncertainty, particularly when dealing with responses that are ambiguous or provide only partial insights. To overcome this limitation, we introduce Conversation-Aware Bayesian Experimental Design (CA-BED). This inference-time probabilistic dialog planning framework combines Bayesian Experimental Design with LLM-based likelihood estimation to refine question selection across multiple conversational exchanges. CA-BED sustains a belief distribution regarding potential hypotheses, forecasts possible responses, and distributes anticipated information gain through a simulated conversation tree. In evaluations involving two structured entity-deduction benchmarks, CA-BED demonstrated an average success rate improvement of 21.8% compared to direct prompting, with similar performance enhancements observed against other information-seeking strategies. These results were achieved with a minimal increase of just 1.8 conversational turns on average relative to direct prompting.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC