Title: MedGym: A Unified Continuous-Time Benchmark for Dynamic Medical Treatment Reinforcement Learning

Abstract:

Reinforcement learning (RL) faces distinct hurdles in the domain of medical treatment recommendation. Patient physiology changes continuously, clinical measurements and interventions occur at irregular intervals, and treatment responses vary significantly from one individual to another. However, current RL formulations and simulated environments largely rely on discrete-time Markov Decision Process (MDP) or Partially Observable MDP (POMDP) abstractions, which assume fixed or pre-specified decision intervals. Consequently, it remains challenging to assess whether RL algorithms can effectively manage time-interval-dependent disease progression, personalized treatment outcomes, and safety protocols between consecutive measurement points.

To bridge this gap, we present MedGym, a benchmark environment designed for dynamic treatment recommendation. MedGym employs Physics-Informed Neural Networks to construct a configurable medical RL benchmark derived from clinical data, modeling longitudinal patient evolution within a continuous-time framework. This benchmark accommodates both offline and online RL approaches, allowing for direct comparisons between discrete-time and continuous-time methods under conditions of irregular treatment timing and patient-specific dynamics. Furthermore, MedGym facilitates evaluation from clinically relevant angles, such as personalization, trajectory-level safety, and the performance disparity between model-based offline learning and online deployment. By offering a standardized and configurable platform for continuous-time dynamic treatment, MedGym seeks to enable more realistic and insightful evaluations of medical RL methods.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC