Title: Pool-Select-Refine: Allocation-Aware Generative Dataset Distillation with Soft-Label-Guided Latent Refinement

Abstract:

Diffusion-based dataset distillation has recently gained traction as a promising approach for compressing extensive datasets into smaller, synthetic representations. By capitalizing on pretrained generative priors, these techniques can generate realistic, class-conditional examples more efficiently than conventional matching-based methods. Nevertheless, the majority of current diffusion-based approaches rely on a rigid “Generate-and-Use” paradigm. In this model, generated samples are immediately accepted as the final distilled set, constrained by a fixed images-per-class quota. This approach tightly binds candidate generation to final budget allocation, potentially leading to inefficient use of the limited budget through redundant waste or the inclusion of samples lacking sufficient informativeness.

To address these limitations, we introduce “Pool-Select-Refine,” a novel two-stage framework designed for allocation-aware generative dataset distillation. The first stage involves constructing an over-complete candidate pool rather than relying on a fixed number of generated samples. From this pool, a compact subset is selected to fit the target budget. In the second stage, the chosen samples undergo refinement in the latent space, guided by soft-label supervision from the teacher model. This process enhances semantic alignment while maintaining the integrity of the generative prior. By explicitly decoupling the processes of generation, selection, and refinement, this design facilitates a more effective utilization of the distillation budget.

Our experiments on large-scale and fine-grained image classification benchmarks demonstrate that the proposed framework achieves consistent improvements over existing diffusion-based baselines. These findings indicate that incorporating a curation phase prior to refinement serves as a straightforward yet potent strategy for advancing diffusion-based dataset distillation.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC