Title: Prioritizing Substance Over Procedure: Enhancing Fairness in Conformal Prediction

While conformal prediction (CP) provides model-agnostic uncertainty quantification, the relationship between its application and fairness in subsequent decision-making processes has not been sufficiently investigated. This study shifts the focus from procedural fairness—viewing CP merely as an isolated step—to a comprehensive evaluation of the entire decision-making workflow, thereby assessing substantive fairness, which concerns the equity of final outcomes.

From a theoretical standpoint, we establish an upper bound that breaks down disparities in prediction-set sizes into distinct, interpretable elements. This analysis elucidates how label-clustered conformal prediction can mitigate method-induced inequities. To support large-scale empirical investigation, we propose a novel evaluator that integrates large language models (LLMs) into the loop, effectively approximating human judgments of substantive fairness across various data modalities.

Our experimental results indicate that label-clustered CP strikes a beneficial balance between utility and substantive fairness, with reductions in set-size disparities aligning with our theoretical predictions. Furthermore, we demonstrate empirically that achieving equalized set sizes, rather than merely ensuring coverage, is significantly associated with enhanced substantive fairness. These findings offer practitioners actionable insights for constructing more equitable CP frameworks. The associated code can be accessed at https://github.com/layer6ai-labs/llm-in-the-loop-conformal-fairness.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC