Title: DAR: Deontic Reasoning with Agentic Harnesses

Original: arXiv:2606.05009v1 Announce Type: cross Abstract: Deontic reasoning is the task of answering questions by applying explicit rules and policies to case-specific facts, for example computing tax liability under a statute or determining the outcome of an immigration appeal. A key technical challenge for LLM-based deontic reasoning is that the relevant ruleset can be long and cross-referenced, so models may still fail to locate the rules needed for a particular reasoning step. We introduce Deontic Agentic Reasoning (DAR), an agentic reasoning setup in which the model interacts with the statutes on demand. We evaluate DAR under multiple harnesses on hard subsets of DeonticBench. Across these settings, we find that agentic harnesses can push the frontier on deontic reasoning tasks, but improvements are not uniform: weaker models often degrade on numerical tasks while consuming far more tokens.

Rewrite: Abstract: Deontic reasoning involves resolving inquiries by applying specific regulations and policies to unique factual scenarios, such as calculating tax obligations according to legal statutes or adjudicating immigration appeals. For large language models (LLMs), a significant technical hurdle in this domain is that the applicable rule sets are often extensive and interlinked, leading to potential failures in identifying the precise rules required for specific logical steps. To address this, we propose Deontic Agentic Reasoning (DAR), a framework where the model engages with statutes interactively as needed. We assess DAR using various harnesses on challenging subsets of the DeonticBench dataset. Our findings indicate that while agentic harnesses can advance the capabilities of models in deontic reasoning tasks, the benefits are inconsistent: less capable models frequently experience performance declines on numerical problems despite utilizing significantly higher token counts.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC