Title: Need to Know: Contextual-Integrity-Grounded Query Rewriting for Privacy-Conscious LLM Delegation

Abstract:

As large language models (LLMs) become deeply integrated into daily professional and personal workflows, queries transmitted to cloud-hosted LLMs frequently contain a mix of information critical to the task and sensitive data that is not. Traditional privacy-preserving methods, which rely on type-based Personally Identifiable Information (PII) redaction, are often context-agnostic. This approach can lead to two significant drawbacks: the unnecessary exposure of untyped sensitive context and the excessive removal of text segments that are essential for generating an answer. To address these challenges, we reframe privacy-preserving query rewriting through the lens of Contextual Integrity, positing that a data span should only be transmitted if it is strictly necessary for the specific task at hand.

We introduce DelegateCI-Bench, the inaugural task-based Contextual Integrity benchmark designed for privacy-aware delegation. This benchmark consists of 3,167 samples, combining high-quality synthetic data across 11 tasks and 20 task types, real-world user queries derived from WildChat, and a specialized medical challenge set featuring dense sensitive information. Leveraging this benchmark, we propose a Contextual Integrity (CI)-guided reinforcement learning framework. This framework transforms essential and non-essential sensitive spans into verifiable optimization signals, enabling the training of a query rewriter that retains task-critical information while suppressing unnecessary sensitive disclosures. Our experimental results demonstrate that the learned rewriter delivers the optimal balance between privacy and utility, achieving an average utility improvement of up to +10.1 over on-device baselines.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC