Title: InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models

Abstract: While Video Large Language Models (Video-LLMs) deliver impressive results in video comprehension, their reliance on a massive number of visual tokens incurs significant computational costs. Current training-free compression techniques attempt to mitigate this by lowering visual token counts to boost inference speed. However, these approaches typically depend on local similarity between adjacent frames to estimate temporal redundancy or distribute token budgets primarily based on segment duration. Such methods are vulnerable to noise at the frame level and overlook the uneven distribution of information found in natural videos. To overcome these limitations, we introduce InfoMerge, a training-free strategy for visual token compression that enhances token efficiency through precise redundancy assessment and content-sensitive budget distribution.

Our approach features the Temporal Fingerprint Difference, a second-order method for estimating temporal redundancy at the segment level. This technique captures the structural similarities of tokens occupying identical spatial positions across frames within a specific segment. Additionally, we propose Content-Aware Budget Allocation (CABA), a mechanism that dynamically assigns token quotas to segments based on their distinctiveness and the richness of their representation, measured via spectral entropy. By minimizing the retention of redundant static areas and prioritizing information-dense segments, InfoMerge optimizes the use of constrained token budgets without compromising performance. Comprehensive evaluations demonstrate that InfoMerge offers superior efficiency-accuracy balance across various benchmarks and model architectures, with benefits becoming more evident under high compression ratios. For instance, when applied to LLaVA-OneVision-7B, InfoMerge preserves 98.8% of the baseline average performance while cutting visual tokens by 85% and accelerating the prefill phase by 4.24 times.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC