Title: Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings

Abstract

The growing incorporation of renewable energy into electrical grids, especially within structures utilizing photovoltaic (PV) arrays and energy storage solutions, has significantly complicated energy management. The combination of fluctuating power output, shifting electricity rates, and a proliferation of components—such as heat pumps and PV installations—has rendered system operations increasingly difficult. Consequently, there is a rising need for advanced control and optimization strategies, including data-driven approaches like reinforcement learning. Although deep reinforcement learning (DRL) offers a promising avenue for optimizing building performance in dynamic and complex settings, its "black-box" characteristics hinder user confidence and widespread implementation.

This study introduces a framework for explainable deep reinforcement learning (XRL) tailored for energy management in residential settings. We validate this approach using both synthetic datasets and real-world data sourced from the Living Lab Energy Campus (LLEC) at KIT. We trained and evaluated both on-policy and off-policy DRL agents utilizing an enriched state space. This expanded context includes real-time metrics (such as demand, PV generation, battery power, and state of charge), external factors (including dynamic electricity prices and local weather conditions), temporal markers (calendar and holiday indicators), and forecasts for both demand and pricing.

Our experimental outcomes reveal that on-policy algorithms, specifically Advantage Actor Critic (A2C) and Proximal Policy Optimization (PPO), surpass off-policy techniques regarding policy stability and cumulative rewards. To elucidate the learned control strategies, we applied post-hoc interpretation methods. The results confirm that the XRL framework not only achieves cost savings via optimized battery management but also delivers clear, actionable transparency into the agent’s decision-making logic.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC