Title: ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents

Abstract:

The exponential growth of scholarly output has created an urgent need for automated deep research (DR) systems; however, the accurate assessment of these systems remains a significant challenge. Current evaluation frameworks typically suffer from two primary limitations: they tend to prioritize retrieval tasks at the expense of high-level planning and reasoning, and they generally emphasize general-purpose domains rather than the academic sectors where DR agents are most applicable. To bridge these critical gaps, we present ADRA-Bank, a novel modular benchmark designed specifically for Academic DR Agents.

Built upon a foundation of academic literature, ADRA-Bank comprises a human-annotated dataset consisting of 200 instances spanning ten distinct academic domains, encompassing both research and review papers. To effectively evaluate these agents, we introduce the ADRA-Eval framework, a modular evaluation paradigm that capitalizes on the intricate structure of academic texts to measure essential capabilities such as planning, retrieval, and reasoning. This framework operates using two complementary assessment modes: an end-to-end evaluation tailored for full \task agents, and an isolated evaluation designed for foundational Large Language Models (LLMs) intended to serve as agent backbones.

Our findings highlight disparities in agent performance. While certain agents demonstrate specialized proficiency, they frequently encounter difficulties with multi-source retrieval and maintaining consistency across different fields. Furthermore, our analysis indicates that enhancing high-level planning is the key to unlocking the full reasoning potential of foundational LLMs when used as backbones. By identifying these specific failure modes, ADRA-Bank serves as a diagnostic instrument aimed at fostering the creation of more dependable automated academic research assistants.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC