Title: Streamlining Safe Exploration in Reinforcement Learning with Accessible Shielding

Abstract

A persistent hurdle in Reinforcement Learning (RL) is safe exploration, a process designed to stop agents from taking dangerous actions as they navigate their surroundings. One established solution is shielding, a technique that leverages domain knowledge—specifically an environment model—to determine which actions are safe. Despite its proven effectiveness, shielding has not been widely adopted in RL communities. This gap exists largely because there is no convenient end-to-end infrastructure linking formal shield synthesis with standard RL frameworks. Consequently, implementing shielding usually demands specialized knowledge in formal methods and significant engineering resources, placing it outside the routine workflow for most RL developers.

To bridge this divide, we have upgraded our shield synthesis tool, Tempest, to serve as a practical backend for safe RL systems. Our primary contribution is tempestpy, a Python library that embeds Tempest-based shield synthesis directly into the Gymnasium API. This integration enables researchers to synthesize and deploy shields seamlessly within existing RL pipelines, significantly reducing the entry barrier. By doing so, we transform formal safe-exploration methodologies into a practical resource for RL practitioners.

Furthermore, we have expanded Tempest’s algorithmic capabilities to generate sound shields for stochastic multiplayer games, ensuring that formal safety guarantees are maintained. We present a complete end-to-end demonstration of this workflow and conduct evaluations comparing shielded and unshielded RL performance across various environments. To assist with modeling, we provide symbolic models for MiniGrid and introduce MiniGridSafe, a suite of playground environments crafted to make shielding accessible and experimentally transparent. MiniGridSafe builds upon MiniGrid by adding safety-focused scenarios characterized by probabilistic transitions and multiple agents, allowing for the investigation of complex safety dynamics in an intuitive and straightforward manner.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC