Title: Rendering Expert Reasoning Accessible via Self-Distillation

Abstract:

Enhancing the reasoning proficiency of large language models (LLMs) generally depends on two factors: the model’s capacity to sample a correct answer for reinforcement learning or the presence of a more powerful model capable of solving the task. Yet, numerous complex problems remain unsolvable for even the most advanced frontier models, which blocks the generation of valid training signals. While utilizing high-quality human expert solutions offers a viable alternative, simple imitation of this data is ineffective because such solutions are fundamentally out-of-distribution. Expert answers are often didactic, featuring implicit reasoning gaps designed for human comprehension rather than for computational models. Additionally, since obtaining high-quality expert solutions is costly, there is a critical need for training methods that are both generalizable and sample-efficient.

To address these challenges, we introduce Distribution Aligned Imitation Learning (DAIL), a two-stage self-distillation approach. DAIL bridges the distributional divide by first converting expert solutions into comprehensive, in-distribution reasoning traces. It then employs a contrastive objective to direct the learning process toward expert insights and methodologies. Our results demonstrate that DAIL can utilize fewer than 1,000 high-quality expert solutions to secure pass@128 improvements of up to 31% on Qwen2.5-Instruct and Qwen3. Furthermore, this method doubles reasoning efficiency and facilitates generalization to out-of-domain tasks.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC