Title: ToolFG: Advancing Fine-Grained Image Classification with Robust Grounding

Abstract: Fine-grained image classification (FGIC) is a field with widespread utility and considerable research interest. This study introduces \textbf{ToolFG}, a pioneering framework that integrates tools into Multimodal Large Language Models (MLLMs) specifically designed for FGIC tasks. By allowing MLLMs to autonomously and adaptively employ external tools during their reasoning phases, ToolFG facilitates active image interaction and the gathering of verifiable visual evidence. This approach enhances the \textit{reliability} and \textit{well-grounded nature} of distinctions between highly similar classes. To endow the model with these capabilities, we introduce a novel \textbf{MCTS-guided tool-use knowledge distillation mechanism} that extracts relevant tool-use and FGIC-specific knowledge from sophisticated proprietary MLLMs to inform training. Additionally, we present a \textbf{model-tool co-evolution mechanism} that simultaneously optimizes the toolset and the model’s policy for using these tools, steering both toward a specialized, mutually adapted configuration for FGIC. Comprehensive experimental results validate the efficacy of our proposed framework.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC