Title: PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning

Abstract

There is frequently a critical window between the initial indication of danger and the actual occurrence of an accident, a period during which intervention can still prevent harm. Video-capable multimodal large language models (MLLMs) have the potential to function as continuous safety monitors, issuing alerts within this crucial timeframe. However, existing benchmarks fail to evaluate this specific capability; they typically depend on static inputs, neglect timing precision, and do not account for false positives in safe scenarios. To address this gap, we introduce PaSBench-Video, a benchmark comprising 740 videos—481 depicting risks and 259 showing no risk—spanning four sectors: driving, healthcare, industrial production, and daily life. Each risk video is annotated with frame-level markers for both the onset of risk and the accident boundary. To succeed, a model must analyze the video causally, generating warnings that are accurate in both content and timing.

Our evaluation of 13 MLLMs reveals that no model surpasses 20.0% on our most stringent metric. Furthermore, there is a strong positive correlation (Pearson = 0.64) between recall and the false-positive rate, indicating that improved detection capabilities come at the expense of triggering warnings on the majority of safe clips. Performance varies significantly by domain: models demonstrate moderate recall with low false-positive rates in daily life contexts, where risks are naturally anomalous. In contrast, they struggle in driving scenarios, where hazardous and routine scenes appear similar, leading to indiscriminate alerts. These findings suggest that current models tend to rely on general scene activity cues rather than reasoning about emerging threats.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC