Title: Establishing a Foundational Model for Wearable Motion Data in Mental Health Studies

Abstract

Smartwatches ubiquitous in the consumer market routinely capture wearable movement data, providing a rich resource for mental health research by revealing nuanced, time-dependent behavioral patterns. However, the advancement of foundation models specifically tailored for health wearables has lagged significantly behind progress in clinical image and text processing. To address this gap, we engineered transformer architectures utilizing patch embeddings and employed self-supervised masked autoencoder pretraining on minute-by-minute, week-long actigraphy sequences to create and assess the Pretrained Actigraphy Transformer (PAT).

PAT serves as an open-source foundation model for wearable movement time series, integrating long-term temporal modeling, psychiatric outcome assessment, and reproducibility using public datasets. Trained on data from 21,538 participants in the U.S. National Health and Nutrition Examination Survey (NHANES), a nationally representative cohort, PAT consistently surpassed non-foundation model baselines across various mental health prediction tasks. These tasks included forecasting the use of benzodiazepines and selective serotonin reuptake inhibitors (SSRIs), as well as identifying depression and sleep irregularities.

Notably, during the prediction of benzodiazepine usage, PAT achieved the most significant gains over standard deep learning time-series models, recording a 55.6% improvement over Long Short-Term Memory (LSTM) networks, a 21.4% gain over 1-D Convolutional Neural Networks (CNNs), and a 14.8% increase over ConvLSTMs. In addition to superior predictive performance, PAT generates interpretable attention maps that pinpoint specific daily activity periods crucial for clinical predictions, thereby enhancing model transparency and offering potential clinical insights. These findings indicate that PAT represents a scalable, adaptable, and easily deployable solution designed to help researchers and clinicians extract deeper clinical insights from wearable sensor data.

GitHub: https://github.com/njacobsonlab/Pretrained-Actigraphy-Transformer/

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC