Title: Estimating $f$-Divergences via Rank-Based Statistics

Abstract: This paper presents a novel method for approximating $f$-divergences using rank statistics, a technique that bypasses the need for explicit density-ratio estimation by operating directly on the distribution of ranks. By introducing a resolution parameter $K$, we transform the discrepancy between two univariate distributions, $\mu$ and $\nu$, into a rank histogram defined over the set ${ 0, \ldots, K}$. The deviation of this histogram from a uniform distribution is quantified using a discrete $f$-divergence, thereby producing a rank-statistic divergence estimator. We demonstrate that this estimator is monotonic with respect to $K$, consistently serves as a lower bound for the true $f$-divergence, and achieves quantitative convergence rates as $K \to \infty$, provided there is mild regularity in the quantile-domain density ratio. To extend this framework to high-dimensional datasets, we propose the sliced rank-statistic $f$-divergence, which averages the univariate construction across random projections, and we also establish convergence properties for this sliced limit. Furthermore, the study derives finite-sample deviation bounds and proves asymptotic normality for the estimator. The efficacy of this approach is empirically validated through benchmarking against neural baselines and by demonstrating its utility as a learning objective in generative modeling tasks.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC