Title: AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination

Abstract:

Eye tracking plays a critical role in smart glasses by offering visibility into user attention for ambient intelligence applications. Nevertheless, the majority of current eye-tracking technologies depend on active infrared (IR) illumination, which imposes significant practical limitations on all-day outdoor usage due to high power demands. This study explores the viability of using passive IR cameras exclusively—eliminating the need for active IR light sources—to achieve reliable pupil detection in unconstrained outdoor settings where natural sunlight acts as the only illumination.

To facilitate this research, we present AmbientEye, a comprehensive dataset comprising 2,606,225 eye images gathered from 35 participants across 19 nations. The data was recorded outdoors in natural sunlight using two distinct off-axis camera setups and two different sun-orientation scenarios. We ensured high-quality pupil annotations by initially employing SAM2 for automatic segmentation, followed by manual refinement by human annotators.

We evaluated a state-of-the-art pupil segmentation algorithm on our new dataset and contrasted its results with performance metrics from existing datasets that utilize controlled IR illumination. The findings indicate a significant decline in segmentation accuracy, dropping from 0.928 on controlled IR datasets to 0.767 on AmbientEye. This disparity underscores the difficulties inherent in ambient-light conditions and establishes AmbientEye as a pioneering benchmark for this largely unexplored yet highly practical eye-tracking context.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC