Title: Enhancing Robustness Through Dynamic Token Selection in Distribution-Aligned Self-Distillation

Abstract: Self-distillation boosts learning efficiency by transforming reference answers into training data that aligns more closely with the model’s internal distribution. Nevertheless, these references often impose significant stylistic biases, leading generative models to mimic surface-level characteristics rather than acquiring meaningful reasoning strategies. Our analysis reveals that rewritten datasets are populated with numerous high-perplexity (PPL) tokens originating from two divergent causes: beneficial logical corrections that enhance knowledge, and detrimental stylistic drift resulting from reference imitation. Equally weighting these tokens can disturb the base model’s inherent distribution, thereby impairing performance, particularly on complex reasoning challenges. To mitigate this issue, we introduce Distribution-Aligned Self-Distillation (DASD). This approach employs an answer-aware reference model to produce candidate tokens, which are then dynamically filtered based on the confidence levels of the base model. DASD retains tokens that convey valuable logical insights while filtering out style-related noise that misaligns with the data distribution. Evaluations across benchmarks for mathematics, code generation, and commonsense reasoning demonstrate that DASD consistently surpasses strong baselines, lowers the occurrence of high-PPL tokens, and enhances robustness across tasks with varying levels of difficulty.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC