Title: Establishing the Energy Baseline: Direct Assessment and Replay Buffer Bias in SAC-Driven HVAC Management via sbsim

Abstract

This study determines the energy floor—the lowest possible operational cost dictated by action space limitations—for HVAC systems controlled by Soft Actor-Critic (SAC) algorithms within the sbsim calibrated building simulator. By conducting minimum-action experiments, we establish a direct measurement of this floor at USD 35.51 per day. This cost is primarily driven by continuous electrical loads, which account for USD 35.44 (99.8% of the total), while gas consumption remains negligible.

In comparison, the standard SAC baseline, which utilizes transitions from a schedule-policy replay buffer for initialization, converges to a daily cost of USD 37.18. This figure sits 4.7% above the identified energy floor. Our analysis reveals that the initialization of the replay buffer is the principal cause of this sub-optimality. When training is initiated from an empty buffer, the daily cost drops to USD 35.57, thereby closing 96% of the performance gap.

Further exploration into parameter adjustments shows that widening the supply water temperature range by 10 K results in minimal savings of just USD 0.03 per day. Any attempt to expand this range further leads to violations of physical constraints. Additionally, we detected a coupling effect in the discount factor (gamma_eff = 0.891), which drastically reduces the effective planning horizon from 8.3 hours to merely 46 minutes. This phenomenon represents a widespread issue across the benchmark that requires immediate audit.

Comprehensive ablation studies focusing on planning horizons, reward weights, and observation enrichment indicate that all configurations utilizing pre-filled buffers consistently cluster within a narrow band of 0.7% (ranging from USD 37.18 to USD 37.42). This consistency demonstrates that the binding constraint on performance is the minimum power requirement of the equipment, rather than the specific design of the algorithm.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC