Title: Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design

Abstract:

Creating high-performance neural networks for emerging tasks necessitates a delicate equilibrium between optimization precision and search efficiency. Existing approaches struggle to maintain this balance; neural architectural search demands prohibitive computational resources, whereas model retrieval typically produces suboptimal static checkpoints. To address this challenge, we conceptualize the performance improvements resulting from fine-grained architectural adjustments as "edit-effect evidence" and assemble evidence graphs derived from previous tasks. Through a retrieval-augmented model refinement framework, our proposed M-DESIGN method dynamically integrates historical evidence to identify modification paths that are nearly optimal. This approach incorporates an adaptive retrieval mechanism capable of rapidly calibrating the shifting transferability of edit-effect evidence sourced from various origins. Furthermore, to mitigate the impact of out-of-distribution shifts, we employ predictive task planners that extrapolate potential gains from multi-hop evidence, thus lessening the dependency on exhaustive repositories. Leveraging a model knowledge base comprising 67,760 graph neural networks across 22 datasets, our extensive experiments reveal that M-DESIGN consistently surpasses baseline methods. Notably, it achieves the best performance within the search space in 26 out of 33 scenarios, even under stringent computational budgets.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC