Title: Supervising Fixed-Wing UAV Commands via HJB-Inspired Risk Filtering and Autopilot-Preserving Residual Q-Learning

Abstract:

Fixed-wing unmanned aerial vehicles (UAVs) must maintain precise airspeed, altitude, and heading references despite disturbances such as turbulence, gusts, and crosswinds. Because these control channels are coupled, adjusting one variable can inadvertently degrade performance in another. While classical autopilots excel at stabilizing the airframe, they struggle to adapt during complex maneuvers, such as aggressive turns in hard crosswinds. Conversely, reinforcement learning (RL) policies that act directly on control surfaces introduce significant exploration risks at the actuator level.

To address these challenges, this study proposes a learned supervisor positioned above an unchanged autopilot, rather than integrated within it. This supervisor selects a residual command from a finite, bounded action set targeting airspeed, altitude, and heading. Before these modified references reach the autopilot—the sole controller interfacing with the actuators—they are projected into an admissible command envelope.

The novelty of this approach lies in the selection mechanism for the residual. Candidates are evaluated using a semi-discrete value-iteration critic, inspired by the Hamilton-Jacobi-Bellman (HJB) equation. These candidates are then ranked based on a no-op-relative Hamiltonian advantage and filtered through a finite-action shield derived from control-Lyapunov and control-barrier principles, ensuring a no-op fallback is always available.

Benchmarking was conducted on a shared 12-state runtime environment with the plant, autopilot, and actuator models held constant, allowing for a direct package-level comparison. The results demonstrate that the HJB residual method reduces mean RMS path-tracking error to 44.809 meters. This represents a significant improvement over the baseline autopilot, which recorded an error of 338.617 meters (an 86.77% reduction), and over a tabular-Q residual method, which achieved 88.809 meters (a 49.54% reduction).

While the performance gains are most pronounced in scenarios where the baseline method fails most severely, this improvement is accompanied by a measured increase in airspeed error, indicating that no single method dominates across all metrics. This paper presents the complete design of this autopilot-preserving residual command-supervision framework, alongside a transparent reporting of its associated trade-offs.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC