Title: Enhancing Diversity and Quality via Base-Aligned Model Collaboration

Abstract:

While alignment techniques have significantly enhanced the output quality of large language models (LLMs), they often do so by reducing diversity, resulting in highly repetitive generations, particularly in open-ended tasks. To address this trade-off, we introduce Base-Aligned Model Collaboration (BACo), an inference-time framework that operates at the token level to dynamically merge a base LLM with its aligned version. By leveraging uncertainty and content-based signals, BACo utilizes specific routing strategies to select the optimal model for decoding each token. Unlike previous methods that prioritize diversity at the cost of quality, or those necessitating costly decoding processes or post-training adjustments, BACo delivers both high diversity and quality in a single pass without additional training. This approach also provides robust controllability. We present a suite of effective routing strategies and test them across three open-ended generation tasks, evaluating performance using 13 distinct metrics for diversity and quality. Our results show that BACo consistently outperforms current state-of-the-art inference-time baselines. The most effective router yields a joint improvement of 21.3% in diversity and quality, a finding corroborated by human evaluations. These findings indicate that collaborating between base and aligned models offers a potent and manageable solution for balancing the diversity-quality spectrum.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC