Title: Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments

Abstract:

As autonomous unmanned aerial vehicles (UAVs) increasingly depend on reinforcement learning (RL) for navigation, they face significant vulnerabilities to global navigation satellite system (GNSS) spoofing. Such attacks can cause out-of-distribution shifts in observations, leading to corrupted value estimation and diminished mission performance. While current robust RL methods often enhance resilience against specific attack vectors, they frequently lack generalization capabilities when confronted with novel threats not seen during training.

To overcome this gap, we introduce a curriculum-guided adaptation framework. This approach progressively subjects a robust policy to gradient-based adversarial observation perturbations of escalating intensity. Crucially, it aligns temporal-difference (TD) error distributions across these curriculum stages. Instead of tailoring the policy to a single attack model, our method maintains TD-error consistency to foster transferability across various attack conditions. We also establish a TD-space generalization certificate, which demonstrates that if the TD-error distribution caused by a test-time attack remains sufficiently similar to that of the final curriculum stage, the resulting drop in performance is mathematically bounded.

We validated this framework within a UAV deconfliction scenario featuring dynamic 3D obstacles, testing it against previously unseen fixed and dynamic GNSS spoofing attacks. Under fixed spoofing conditions, our curriculum-adapted policy attained near-perfect mission success rates, significantly outperforming standard and robust RL baselines, which achieved success rates between 20% and 56%. Furthermore, in scenarios involving dynamic obstacle-luring spoofing, the proposed method secured the highest episodic rewards and decreased the number of steps required to complete missions by up to 45%, even as aerial traffic densities increased.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC