Title: A High-Performance Framework for Streaming Video Analysis via Agentic Control

Streaming video processing necessitates managing fluctuating information density within rigid latency limits. However, current approaches often rely on static mechanisms, such as fixed memory compression or the exclusive use of a single model. This rigidity creates a difficult trade-off: lightweight models struggle with complex inquiries, whereas continuously running resource-intensive models breach real-time requirements and unnecessarily complicate straightforward tasks.

Instead of predetermining these operational choices, we introduce R3-Streaming (Remember, Respond, Reason). This framework treats streaming video understanding as a cascaded control challenge. For every query, the system sequentially compresses its memory, assesses whether a response is ready, and directs computational resources. This ensures that each subsequent decision is informed by increasingly refined information states.

To enhance this pipeline, we developed an age-aware forgetting strategy for memory compression, demonstrating that aggressively discarding older frames can significantly boost performance. Additionally, we propose TB-GRPO, a target-balanced reinforcement learning objective designed for compute routing. This method directs difficult queries to more powerful models while effectively avoiding mode collapse.

Comprehensive evaluations show that R3-Streaming sets a new state-of-the-art standard for streaming Multimodal Large Language Models (MLLMs). It achieved scores of 57.92 on OVO-Bench and 76.36 on StreamingBench, all while decreasing visual token consumption by 95 to 96 percent.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC