Title: OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models

Abstract:

Electroencephalography (EEG) facilitates a broad spectrum of brain-computer interface (BCI) applications, extending from monitoring brain states to enabling interactions with large language models (LLMs). While EEG foundation models are rapidly evolving, the field currently suffers from fragmented evaluation standards caused by inconsistent task protocols and heterogeneous datasets. To address this challenge, we present OmniEEG-Bench, a comprehensive benchmark and roadmap for downstream tasks designed specifically for EEG foundation models (FMs).

This framework categorizes model assessments into six distinct task families: (i) signal reliability, (ii) biometrics and disease detection, (iii) consciousness and state analysis, (iv) cognition and emotion processing, (v) naturalistic stimulus decoding, and (vi) motor and interaction tasks. Notably, this structure introduces a new generation of tasks that have not been systematically benchmarked in previous EEG FM research. OmniEEG-Bench establishes a standardized approach to model deployment, task definitions, and performance metrics via a unified task-card specification. It integrates 54 EEG datasets, ensuring consistent evaluation protocols across the board.

We evaluated 10 representative EEG foundation models and published a leaderboard reflecting these diverse evaluation settings. Our findings indicate that both the diversity of pretraining datasets and the size of the model are significantly correlated with improved average rankings across datasets, demonstrating clear scaling-law behavior in EEG foundation models (see Figure 1). These results imply that enhancing the performance of EEG foundation models necessitates not only larger architectural scales but also access to broader and more varied pretraining data. The source code for the benchmark is publicly accessible at https://github.com/ncclab-sustech/omni-eegbench.git.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC