Title: Dr. DocBench: A Rigorous Benchmark for Expert-Level and Complex Document Parsing

Abstract

Vision-language models (VLMs) and document processing systems rely heavily on robust document parsing and recognition capabilities. However, current benchmarks for Optical Character Recognition (OCR) and document parsing are becoming increasingly constrained by their scope and lack of challenge. Many existing resources concentrate on standard document genres or uniformly sampled pages where contemporary parsers already demonstrate high proficiency, while providing insufficient annotation for specialized domain structures such as musical notation, chemical formulas, intricate tables, and multi-page layouts.

To address these gaps, we present Dr. DocBench, a difficulty-aware benchmark designed specifically for expert-level document parsing. Derived from a vast multilingual book corpus, Dr. DocBench covers 52 BISAC subject domains. It employs a parser-failure-based sampling strategy to identify and select documents that pose significant challenges for multiple state-of-the-art systems. The dataset comprises 4,514 annotated pages extracted from long documents, which average approximately 100 pages in length. These pages feature 65,000 high-quality annotations at both the page and block levels, detailing layout, reading order, hierarchical relationships, and domain-specific visual elements.

Our evaluation of pipeline-based parsers and general-purpose VLMs indicates that high performance on existing benchmarks does not necessarily translate to success in expert-level document parsing tasks. Our analysis uncovers significant errors across various subjects, content types, and structural attributes. These findings position Dr. DocBench as a comprehensive testbed for diagnosing limitations and driving advancements in document intelligence.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC