Title: Can Vision Models Truly Forget? Mirage: Representation-Level Certification of Visual Unlearning

Abstract:

While machine unlearning within Vertical Federated Learning (VFL) is garnering increasing attention, current methodologies rely exclusively on output-level metrics to verify the erasure of data. This study questions the validity of such claims by presenting Mirage, a novel framework for auditing at the representation level. Mirage integrates four distinct diagnostic tools: Layer-Wise Recovery Analysis, Feature Separability Scoring, Centered Kernel Alignment (CKA), and Linear Probe Recovery (LPR).

We evaluated Mirage against seven baseline methods across seven datasets, adhering to recent VFL unlearning protocols. The analysis yielded three critical insights:

1. The Forgetting Gap: Models that satisfy output-level certification often retain significant class structures within their internal representations. Specifically, LPR scores in these models exceeded those of a retrained baseline by as much as 15.4 points. Furthermore, CKA metrics indicate that these models maintain a structural proximity to the original data that is greater than their distance to a retrained reference. Additionally, separability scores highlight ongoing geometric discrimination capabilities.
2. The Unlearning Trilemma: Our results demonstrate that no current method can simultaneously optimize for high utility, output-level forgetting, and representation-level forgetting.
3. Class-Sample Asymmetry: There is a marked disparity between forgetting classes versus individual samples. Class-level forgetting leaves substantial representational footprints, with LPR reaching up to 97%. In contrast, sample-level forgetting yields results indistinguishable from random chance, with LPR hovering around 50%. Layer-wise analysis confirms that residual class information endures across various network depths.

These findings underscore the necessity for federated unlearning research to adopt evaluation standards that are sensitive to representation-level changes.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC