Title: Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation

Abstract:

This paper introduces Dynamic Meta-Metrics (DMM), a novel framework for assessing machine translation quality that generates combinations of established metrics conditioned on the source sentence. Unlike traditional approaches that depend on fixed ensembles or language-specific weights, DMM dynamically adjusts its metric aggregation strategy according to the characteristics of the source segment. We investigate two conditioning strategies: a "hard" approach that assigns an interpretable combiner to each cluster, and a "soft" extension where weights shift continuously based on source-cluster responsibilities. Our evaluation utilizes pairwise agreement metrics at both the system and segment levels, leveraging data from the WMT Metrics Shared Task across various language pairs. The results demonstrate that combinations based on Multi-Layer Perceptrons (MLP) consistently surpass those using linear or Gaussian process-based ensembles. Furthermore, the implementation of soft conditioning provides additional performance improvements over standard linear models.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC