Title: DiTTo: A Scalable, Order-Aware Agent for Comprehensive Image Restoration

Abstract:

In practical scenarios, images often endure multiple forms of degradation simultaneously, and the sequence in which these impairments are corrected significantly influences the ultimate restoration outcome. This reality has spurred the development of agent-based image restoration (IR) systems, which employ vision-language models to orchestrate a collection of pre-trained restoration experts. Nevertheless, current training-driven agents face two major limitations. First, they demand $\mathcal{O}((N^{\mathbf{D}})^{2})$ calls to restoration experts for each image to generate the Optimal Restoration-action Trajectory Dataset (ORTD), where $N^{\mathbf{D}}$ represents the quantity of degradation types within the universe $\mathbf{D}$. Second, they tightly couple the agent’s training to a static set of restoration experts, meaning that incorporating new experts necessitates a complete retraining process.

To address these challenges regarding efficiency and extensibility, we introduce DiTTo, a novel order-aware image restoration agent framework composed of the DiTTo Simulator and the DiTTo Agent. The DiTTo Simulator leverages $\cup$S-IR for single-step simulation of restoration actions and AiO-IQA for quality assessment after each action. This approach drastically reduces the cost of constructing the ORTD to $\mathcal{O}(N^{\mathbf{D}})$ simulator calls per image. Subsequently, the DiTTo Agent undergoes Supervised Fine-Tuning (SFT) using the ORTD generated by the simulator, followed by Order-aware Restoration Alignment (ORA). ORA independently aligns degradation identification, the ordering of restoration actions, and the output format. This design facilitates plug-and-play scalable extensibility, as integrating a new restoration expert only requires updating the lightweight ORA stage. In evaluations on the MiO-100 dataset, which includes images with up to five simultaneous degradations, the DiTTo Agent demonstrates state-of-the-art performance in multi-degradation restoration quality compared to existing agent-based IR methods.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC