Title: ProactiveLLM: Enabling Active Interaction in Streaming Large Language Models

Abstract:

Conventional Large Language Models (LLMs) typically operate on a read-then-generate framework, which introduces avoidable delays and computational overhead. While streaming LLMs address this by generating outputs concurrently with input reception, they often lack the capability to determine the optimal moments for interaction. Current solutions generally depend on pre-defined interaction schedules or expensive external alignment data, including timing annotations, reasoning traces, or guidance from superior teacher models. To overcome these limitations, we introduce ProactiveLLM, a framework that facilitates active interaction by utilizing the model’s internal states to inform decision-making.

ProactiveLLM teaches the model to assess semantic completeness from partial inputs via two distinct training strategies: mask-based streaming modeling and synchronized privileged self-distillation (SPSD). The first strategy employs monotonic random masking during training to mimic the gradual unveiling of streaming data, allowing the model to capture local semantic dependencies from incomplete input perspectives. The second strategy aligns the student’s view, based on partial context, with a teacher’s view, based on full context, both generated by the same evolving model. This approach uses full-context evidence to steer the student’s comprehension when observations are incomplete.

By combining these mechanisms, ProactiveLLM generates endogenous sufficiency signals without the need for external teachers or manual annotations. This creates a flexible foundation for integrating various decision heads in a plug-and-play manner. Comprehensive tests across both text and speech streaming applications demonstrate that ProactiveLLM markedly lowers interaction latency without compromising output quality, thereby confirming its effectiveness in supporting dynamic and proactive interaction. The source code is accessible at https://github.com/EIT-NLP/StreamingLLM/tree/main/ProactiveLLM.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC