Title: OARelatedWork: A Comprehensive Dataset of Related Work Sections Paired with Full Texts from Open Access Repositories

Abstract:

This study presents OARelatedWork, a novel dataset designed to facilitate the generation of related work sections using open-access materials. As the inaugural large-scale multi-document summarization resource dedicated to this specific task, OARelatedWork comprises complete related work sections alongside the full texts of the papers they cite. The validation and test subsets are specifically engineered to ensure that every cited document is fully accessible, thereby allowing for rigorous, controlled assessments of full-text-based related work generation. The collection encompasses 94,450 papers and references 5,824,689 unique cited works across various disciplines. Our primary objective with OARelatedWork is to transition the research community away from the current practice of generating partial related work summaries based solely on abstracts, toward the creation of comprehensive sections utilizing all available content.

Through this dataset, we achieve three main contributions. First, we benchmark a diverse array of models, demonstrating that synthesizing extensive full-text contexts remains a significant hurdle for even contemporary Large Language Models (LLMs). Under our statement-level evaluation criteria, the evidence-grounded True rate for GPT-4o-mini declines from 92.9% when using abstracts to 83.8% when processing full texts. Second, we perform an empirical analysis of human writing habits via a human evaluation involving 40 papers and 408 factual statements. We found that authors often make abstractive claims not strictly grounded in localized source texts; consequently, advanced LLMs outperform human baselines in terms of strict, evidence-based factuality. Finally, we execute a detailed meta-evaluation which shows that conventional reference-based metrics are insufficient for assessing long-form structured outputs. To bridge this gap, we propose a robust statement-level evaluation framework.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC