Title: Optimizing Low-Level Quadcopter Control via Dynamic Entropy Tuning in Reinforcement Learning: Balancing Stochasticity and Determinism

Abstract

This study investigates the influence of dynamic entropy adjustment within Reinforcement Learning (RL) frameworks designed to train stochastic policies, evaluating their efficacy against methods that develop deterministic policies. While stochastic policies aim to maximize rewards by optimizing an action probability distribution, deterministic policies identify a single, fixed action for each state. The research specifically examines the outcomes of training a stochastic policy using both static and dynamic entropy mechanisms, followed by the execution of deterministic actions for quadcopter control. These results are juxtaposed with those obtained from training a deterministic policy under similar execution conditions. For this analysis, the Soft Actor-Critic (SAC) algorithm serves as the representative for stochastic approaches, whereas the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is employed for deterministic strategies. Empirical data from training and simulation phases indicate that dynamic entropy tuning significantly enhances quadcopter control performance. This improvement is attributed to the mitigation of catastrophic forgetting and the increased efficiency of the exploration process.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC