Title: Alignment-Aware Decoding

Original: arXiv:2509.26169v2 Announce Type: replace Abstract: Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving alignment, typically through training-time or prompt-based interventions. In this paper, we introduce alignment-aware decoding (AAD), a method to enhance model alignment directly at inference. Theoretically, AAD can be interpreted as implicit reward optimization, yet it requires no specialized training beyond the standard DPO setup. Empirically, AAD consistently outperforms strong baselines across diverse alignment benchmarks and model scales. Moreover, in data-constrained settings, AAD can produce high-quality synthetic data to improve alignment under standard decoding, providing a practical solution when labeled data is limited.

Rewrite: Efficiently aligning large language models continues to be a primary hurdle within the field of natural language processing. While preference optimization has gained traction as a robust strategy for refining alignment—usually executed via interventions during training or at the prompt stage—this study presents a novel approach called alignment-aware decoding (AAD). This technique is designed to boost alignment fidelity directly during the inference phase. From a theoretical standpoint, AAD functions as a form of implicit reward optimization but does not demand any specialized training procedures beyond those used in standard Direct Preference Optimization (DPO). Experimental results demonstrate that AAD reliably surpasses competitive baseline methods across a wide range of alignment benchmarks and varying model sizes. Furthermore, in scenarios where labeled data is scarce, AAD generates high-quality synthetic data that enhances alignment performance under conventional decoding, offering a viable solution for resource-constrained environments.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC