Title: Enhancing Interpretability in Driver Attention Analysis via Vision-Language Models

Abstract: While driver gaze is frequently represented as spatial heatmaps, these visualizations often lack interpretability for human analysts, as they fail to identify specific road objects or regions under surveillance and do not clarify the significance of attention shifts. This research investigates whether limited human-grounded supervision can guide a vision-language model (VLM) to generate interpretable descriptions of changes in driver attention. By leveraging selected high-change gaze moments from the Berkeley DeepDrive-Attention dataset, we evaluated zero-shot, one-shot, and LoRA fine-tuned VLM configurations against human-refined reference descriptions and expert ratings. Our results indicate that fine-tuning the model with just 80 expert-refined attention examples significantly enhances performance metrics, including ROUGE-L, METEOR, Entity Alignment F1, and the Human Alignment Score, compared to unsteered VLM outputs. These findings imply that linguistic descriptions can serve as a valuable complement to gaze heatmaps, thereby facilitating human-factors analysis, driver-monitoring reviews, and situation-awareness support by rendering driver attention more accessible.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC