Title: How Hard Can It Be? Hardness-Aware Multi-Objective Unlearning

Original: arXiv:2606.02119v1 Announce Type: cross Abstract: Machine unlearning aims to remove the influence of specific forget training data due to privacy, copyright or bias concerns while maintaining the model performance on the remaining retain data. Existing unlearning algorithms, such as optimizing a weighted combination of losses, have tried to achieve these objectives of improving forget quality and maintaining retain utility. However, they do not guarantee that these objectives can be improved by a specified extent for all forget and retain data. In this work, we address this limitation with a novel and theoretically-grounded approach from a constrained optimization perspective. Firstly, we identify that the hardness of reconciling both objectives can be quantified by the similarity between the forget data and the retain data. Next, we derive an unlearning algorithm (HAMU) with the overall goal of guaranteeing a specified improvement in forget quality while minimizing the retain utility cost/degradation by updating the model weights based on our hardness measure. Our hardness measure also informs users when retain utility degradation is unavoidable, i.e., both objectives cannot be improved simultaneously, and stopping should be considered. Our algorithm is applicable to non-convex models and is easily parallelizable, making it readily deployable in real-world scenarios. We empirically demonstrate HAMU's superior performance over baselines on both image and text datasets using large models. Our code is available at https://github.com/aoi3142/HAMU.

Rewrite: Abstract: Machine unlearning seeks to eliminate the impact of designated "forget" training examples—often due to privacy, copyright, or bias issues—while preserving model accuracy on the "retain" dataset. Current methods, which typically optimize a weighted sum of loss functions, attempt to balance improved forget quality with retained utility. However, these approaches fail to ensure that a specific degree of improvement is achieved across all forget and retain samples. To overcome this shortcoming, we propose a novel, theoretically sound method framed as a constrained optimization problem. We first establish that the difficulty of balancing these dual objectives can be measured by the similarity between the forget and retain data. Building on this insight, we introduce HAMU, an algorithm designed to ensure a predefined improvement in forget quality while minimizing the associated drop in retain utility. HAMU updates model weights according to this hardness metric. Additionally, the metric serves as a diagnostic tool, alerting users when a trade-off is inevitable—meaning both objectives cannot be simultaneously enhanced—and suggesting that the unlearning process be halted. Suitable for non-convex models and highly parallelizable, HAMU is well-suited for practical deployment. Empirical evaluations on large-scale image and text models show that HAMU outperforms existing baselines. The source code is publicly accessible at https://github.com/aoi3142/HAMU.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC