Title: MAdam: Metric-Aware Multi-Objective Adam

Abstract:

Although multi-objective optimization (MOO) is foundational to numerous machine learning tasks, existing MOO solvers—spanning loss-balancing, gradient-balancing, and Pareto-based methodologies—routinely delegate their reconciled update directions to the Adam optimizer. This standard practice, however, creates two distinct discrepancies between the solver’s intended direction and Adam’s actual execution. The first issue, termed a weighting mismatch, arises because Adam’s second-moment denominator mixes the time-varying preference vector with gradient statistics. This entanglement effectively reduces the preference to a historical average, causing distinct Pareto trade-offs to converge toward a near-uniform distribution. The second issue, a geometric mismatch, occurs because Adam’s adaptive metric alters the Euclidean geometry that MOO solvers typically assume, thereby transforming aligned objectives into perceived conflicts.

To address both issues simultaneously, we propose MAdam (Metric-Aware Multi-Objective Adam). MAdam functions as a drop-in wrapper that requires no modifications to either the solver or the optimizer. It works by preconditioning the reconciled direction using the curvature of the scalarized objective, conditioned on the current preferences. When Adam processes this whitened input, its second moment reduces to the identity matrix, ensuring that the final update is driven by the preference-conditioned metric. Experiments across diverse domains, including multi-task learning, Pareto-front recovery, physics-informed neural networks, and medical imaging, demonstrate that MAdam consistently outperforms standard Adam across all solver families.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC