Title: Advancing Sparse Video Comprehension and Reasoning

Abstract:

We introduce \revise (\underline{Re}asoning with \underline{Vi}deo \underline{S}parsity), a multi-turn agent designed for video question answering (VQA). Rather than relying on uniform frame sampling, \revise identifies a concise set of informative frames, preserves a summary as the state across iterations, and terminates early once it achieves sufficient confidence. The framework accommodates proprietary vision-language models (VLMs) through a "plug-and-play" approach and facilitates reinforcement fine-tuning for open-source architectures. For the fine-tuning process, we propose EAGER (Evidence-Adjusted Gain for Efficient Reasoning), an annotation-free reward mechanism comprising three distinct components: (1) Confidence gain: Upon the inclusion of new frames, the system rewards the expansion of the log-odds margin between the correct answer and the most competitive alternative; (2) Summary sufficiency: At the point of answer generation, the model is re-evaluated using only the final committed summary, with success being rewarded; (3) Correct-and-early stop: The system is incentivized to provide accurate answers within a limited number of turns. Experiments across various VQA benchmarks indicate that \revise enhances accuracy while simultaneously decreasing the number of frames processed, interaction rounds, and prompt tokens, thereby showcasing the viability of sparse video reasoning.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC