Title: TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning

Abstract:

The TimeLogic Challenge assesses the capability of models to perform formal temporal-logic reasoning on video content, utilizing 16 distinct operators (such as before, after, until, since, always, and co-occur) presented in both boolean and 4-way multiple-choice formats. Current end-to-end video-language models (VLMs) struggle significantly with this task, performing at levels close to random chance. This deficiency stems from their tendency to treat video as a mere collection of frames, leaving them unable to pinpoint the precise timing of actions.

To address this, we introduce TLG (Temporal-Logic Grounding), a three-tiered system designed to enhance performance. First, the system reconstructs the action timeline for each video by leveraging public source-dataset annotations used to generate the benchmark. It then parses every question into a temporal-logic program and executes it deterministically. Second, in instances where no annotation is available, the system defaults to a powerful open-source VLM. Third, it selectively routes only those question categories where the VLM demonstrates empirical weakness to a state-of-the-art reasoning model.

This approach significantly boosts performance, increasing test accuracy from a baseline of 46.9% for standard VLMs to 71.37% with TLG—a substantial absolute gain of 24.5 percentage points. This result places TLG within three points of the current leaderboard leader. Our extensive ablation studies, which include three model-based timeline-reconstruction variants, reveal that all such variants underperform a holistic VLM. These findings isolate temporal grounding as the critical bottleneck, demonstrating that the availability of real annotations, rather than the use of larger models, is the primary driver of accuracy improvements.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC