Title: Conditional Query Lower Bounds via Occupancy Rashomon Capacity: Auditing Near-Optimal Policies Can Be Exponentially Hard

Abstract

In reinforcement learning scenarios where numerous policies yield near-optimal returns, a post-hoc auditor faces the challenge of differentiating between behaviorally distinct yet return-equivalent strategies. We conceptualize this difficulty using an occupancy-measure counterpart to Rashomon capacity, defined as the metric entropy of the near-optimal occupancy region relative to a specific audited deployment class. Since occupancy measures distinguish behavior only up to equivalence, we frame the auditing process at the level of occupancy classes, differentiating between exact local-query oracles and noisy sample-query oracles.

Our primary finding for exact queries is conditional: if the audited class includes a $2/H$-separated near-optimal packing characterized by $b$-sparse local signatures, then exact local-query auditing necessitates $\Omega(M/b)$ queries. In cases where the packing achieves deployment-class capacity and $b=O(1)$, this requirement escalates to $\Omega(2^{\Hopt^\cF(\eps)})$. We demonstrate a finite discounted hidden-branch MDP that reaches this bound and establish the exact Bayes success law.

For noisy hidden-trigger testing, we derive a mixture lower bound of order $M/\beta$, with $\beta$ representing the per-sample KL signal. This results in a lower bound of $\Omega(2^{\Hopt^\cF(\eps)}/(\rho^2\Delta^2))$ for capacity-order packings where $\beta=O(\rho^2\Delta^2)$. Additionally, we present a static target-recognition information lower bound, an upper bound for verification via transcript-compatible oracle covers, and a canonical occupancy regularizer that reduces the regularized audited capacity when a trusted reference occupancy is present. Controlled benchmarks are used to separate positive instances with sparse signatures from high-capacity negative controls, where exact auditing remains straightforward, while also mapping the noisy-trigger law to continuous-control and visual-RL auditing regimes following post-processing.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC