Title: Moving Past Task-Agnostic Methods: Task-Aware Grouping for Communication-Efficient Multi-Task MoE Inference

Abstract:

While Sparsely Activated Mixture-of-Experts (MoE) models leverage conditional computation to expand their capacity, distributed inference faces significant challenges related to load imbalance caused by routing and high communication costs for cross-GPU expert access. Current placement strategies mitigate these issues by co-locating experts that are frequently activated together. However, these methods typically rely on a single deployment plan derived from globally aggregated routing data, which effectively smooths out the heterogeneous, task-specific co-activation patterns that are the actual drivers of communication overhead in multi-task environments.

Our analysis reveals that expert co-activation is heavily dependent on the task at hand: pairs of experts that are tightly linked within one task family may show no correlation in another. Consequently, optimal deployment requires grouping experts based on task-aware co-activation rather than a task-agnostic average. Guided by this insight, we introduce \emph{Task-Aware Coactivation Grouping} (TACG), a framework for deployment that leverages family-specific dispatch and co-activation data to determine per-expert task-family preferences. TACG reweights the co-activation graph to prioritize intra-family locality during grouping and assigns each expert to a primary GPU while strictly adhering to capacity constraints.

To ensure the static placement remains robust against online workload skew, we also propose \emph{Generic Expert Shared Replication} (GESR). This lightweight complementary approach identifies generic experts that maintain consistently central co-activation profiles, replicates them across a limited number of secondary GPUs, and employs locality- and load-aware selection mechanisms during serving. Evaluations across three prominent open-source MoE models indicate that our framework cuts average communication costs by 31.39\% compared to the baseline, while maintaining an average Jain fairness index of 0.9975. This performance benefit holds even under significant distribution shifts in inference data, consistently surpassing strong baseline methods.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC