Title: LocalSearchBench: Evaluating Agentic Search Capabilities in Real-World Local Life Services

Abstract

The emergence of large reasoning models (LRMs) has empowered agentic search systems to execute intricate, multi-step reasoning across various data sources. Despite this progress, the majority of existing research concentrates on general information retrieval, largely overlooking vertical domains that present distinct difficulties. This study zeroes in on the local life services sector, introducing LocalSearchBench to address its diverse and complex business landscapes. Queries within this field are frequently ambiguous, demanding multi-hop reasoning that spans both merchants and products—a challenge that remains insufficiently tackled.

As the inaugural comprehensive benchmark for agentic search tailored to local life services, LocalSearchBench features a database containing more than 1.3 million merchant records distributed across six service categories and nine major cities. Additionally, it includes 900 multi-hop question-answering tasks derived from authentic user queries, necessitating multi-step logical deduction. To facilitate model interaction, we also engineered LocalPlayground, a consolidated environment that integrates a variety of tools for LRMs.

Our experimental results indicate that even the most advanced LRMs face significant hurdles on LocalSearchBench. The top-performing model, DeepSeek-V3.2, attained a correctness rate of merely 35.60%. Furthermore, most models exhibited deficiencies in completeness (averaging 60.32%) and faithfulness (averaging 30.72%). These findings underscore the critical necessity for specialized benchmarks and domain-specific agent training within the local life services industry. The code, benchmark dataset, and leaderboard are publicly accessible at https://localsearchbench.github.io/.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC