Title: MERIT: A Decentralized Approach to Instruction Tuning via Conflict-Aware Partitioning and Weight Integration

Abstract: Aligning large language models, including multimodal architectures, with varied user intents typically relies on instruction tuning. However, scaling this process to handle heterogeneous data mixtures is often impeded by gradient interference and the high bandwidth costs associated with frequent synchronization. This study investigates whether these two challenges can be simultaneously resolved by training subsets of the mixture independently and reconciling them within parameter space. We establish a local quadratic theory within a shared flat basin, which leads to three key insights: first, weight merging reduces variance by accounting for curvature; second, splitting data along Principal Component Analysis (PCA)-aligned conflict axes maximizes this benefit in high-curvature directions; and third, the merging process functions as spectral filtering with implicit norm regularization. These findings underpin MERIT, a decentralized pipeline designed for merge-ready instruction tuning. MERIT identifies dataset-level gradient conflicts, divides the mixture based on the primary PCA conflict axes, and fine-tunes each segment independently without requiring communication between partitions. The segments are eventually merged using token-weighted averaging. In evaluations on Qwen2.5-VL-3B across 136 Vision-FLAN tasks, MERIT raised the average score across eight benchmarks from 54.3 (achieved by joint training) to 57.0. This methodology also scales effectively to a 7B model trained on a 1.6M-example mixture sourced from 176 distinct datasets, achieving performance that matches or surpasses centralized joint training with negligible additional cost. Furthermore, the approach proves transferable to text-only FLAN tasks. The source code is publicly available at https://github.com/naver-ai/merit.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC