Title: Enhancing Proactivity in Task-Oriented Dialogue

Abstract:

Proactive task-oriented dialogue (TOD), including scenarios like outbound sales, requires agents that can actively investigate user concerns and guide interactions toward a positive outcome within a limited number of turns. However, post-trained large language models (LLMs) tend to be conservative by nature, and reinforcement learning with reward shaping (such as GRPO) faces challenges because it merely re-weights samples from an already passive policy. This paper demonstrates that conditioning on the user’s latent concerns unlocks proactive capabilities that cannot be negated by increased sampling, thereby establishing these concerns as a critical signal during training.

To put this insight into practice, we developed the Cognitive User Simulator. This tool represents each user as a stratified persona, defined by both visible external traits and concealed internal concerns. It generates diverse and realistic interactions while providing per-turn state dynamics that monitor the progress of persuasion. Furthermore, we propose Simulator-Induced Asymmetric-View Policy Optimization, a method that leverages modeled concerns and simulation state transitions to create complementary training objectives: (1) Asymmetric On-Policy Self-Distillation, which transfers concern-aware behaviors from a privileged perspective of the same policy to its deployable, conversation-only view; and (2) State-Transition Policy Refinement...

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC