Title: Deriving Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation

Abstract:

Generating long-form reports in the style of DeepResearch continues to present significant hurdles, primarily because the training and evaluation phases often lack verifiable reward signals. Consequently, rubric-based assessment has emerged as a standard methodology. However, current methods face notable limitations: they either utilize broad, pre-established rubrics that lack adequate detail or depend on manually crafted, query-specific rubrics that are expensive and hard to scale. To address these issues, this study introduces a pipeline designed to train rubric generators that are grounded in human preferences and tailored to specific queries for DeepResearch report generation.

Our approach begins with the creation of a dataset containing DeepResearch-style queries, where human preferences are annotated across pairs of generated reports. We then train the rubric generators using reinforcement learning, employing a hybrid reward function that integrates preference consistency, format validity, and LLM-based rubric evaluation. The effectiveness of these learned rubric generators is assessed through a two-stage evaluation process. Initially, we test them on a held-out set of human-preference data, where they demonstrate a superior ability to distinguish between preferred and rejected reports compared to generic, prompted, or supervised fine-tuning (SFT)-trained rubric alternatives. In the second stage, when these rubric generators serve as reward signals to train DeepResearch systems, they lead to significant performance improvements. These gains are observed across both a straightforward single-agent ReAct framework and a more intricate multi-agent workflow within the DeepResearch Bench.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC