Title: TANDEM: Bi-Level Data Mixture Optimization with Twin Networks

Original: arXiv:2606.04401v1 Announce Type: new Abstract: The capabilities of large language models (LLMs) significantly depend on training data drawn from various domains. Optimizing domain-specific mixture ratios can be modeled as a bi-level optimization problem, which we simplify into a single-level penalized form and solve with twin networks: a proxy model trained on primary data and a dynamically updated reference model trained with additional data. Our proposed method, Twin Networks for bi-level DatA mixturE optiMization (TANDEM), measures the data efficacy through the difference between the twin models and up-weights domains that benefit more from the additional data. TANDEM provides theoretical guarantees and wider applicability, compared to prior approaches. Furthermore, our bi-level perspective suggests new settings to study domain reweighting such as data-restricted scenarios and supervised fine-tuning, where optimized mixture ratios significantly improve the performance. Extensive experiments validate TANDEM's effectiveness in all scenarios.

Rewrite: The performance of large language models (LLMs) is heavily influenced by the diversity of domains represented in their training datasets. This study frames the task of optimizing domain-specific mixture ratios as a bi-level optimization challenge. To address this, we introduce Twin Networks for bi-level DatA mixturE optiMization (TANDEM), a method that reduces the bi-level problem into a single-level penalized formulation. TANDEM employs twin networks comprising a proxy model trained on primary data and a reference model that is dynamically updated using supplementary data. The method evaluates the utility of specific data sources by calculating the divergence between these twin models, subsequently assigning higher weights to domains that yield greater improvements from the additional data. Compared to existing techniques, TANDEM offers broader applicability and robust theoretical guarantees. Additionally, this bi-level framework opens up novel avenues for investigating domain reweighting in contexts such as supervised fine-tuning and data-constrained environments, where the application of optimized mixture ratios leads to substantial performance gains. Comprehensive experimental results confirm the efficacy of TANDEM across all tested scenarios.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC