Title: SANE: Schema-Grounded Natural Language Assessment for Biological Databases

Abstract: While high-throughput microscopy produces vast, organized datasets detailing cellular reactions to drug treatments, retrieving information from these collections usually demands proficiency in SQL. Although large language models (LLMs) present a natural-language interface as an alternative, their propensity for hallucination casts doubt on the trustworthiness of their outputs. To address this, we introduce SANE (Schema-Aware Natural-language Evaluation), a new framework for domain-specific text-to-SQL assessment. SANE utilizes benchmarks that are automatically generated and anchored in real-world, specific experimental structures, thereby making the evaluation process more scalable, systematic, and reproducible.

Our study employs SANE to assess a few-shot LLM, demonstrating that precise query generation is possible without any model training or fine-tuning, provided that constrained schemas are used alongside structured prompting and guardrails. We found that most errors do not result from faulty SQL syntax but rather from ambiguous or underspecified user inputs. These issues typically lead to excessive clarification requests or responses to queries that require disambiguation before answering. Consequently, our findings suggest that few-shot LLMs can deliver reliable database access in well-defined domains when paired with schema-aware prompting techniques.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC