Title: Modeling Empathy Applicability for Broad Health Inquiries

Abstract:

As large language models (LLMs) become increasingly embedded in clinical settings, they frequently struggle to demonstrate clinical empathy—a critical component of effective physician-patient interaction. Current natural language processing (NLP) systems primarily concentrate on retrospectively categorizing empathy within medical responses, providing insufficient support for the proactive modeling of empathy requirements, particularly regarding general health questions. To address this gap, we present the Empathy Applicability Framework (EAF), a theory-based methodology that evaluates patient inquiries by assessing the relevance of emotional responses and interpretations through clinical, contextual, and linguistic indicators.

We have introduced a benchmark dataset comprised of authentic patient queries, which were annotated by both human reviewers and GPT-4o. Our analysis of the subset marked with human consensus reveals a high degree of alignment between human and GPT assessments. To test the efficacy of EAF, we developed classifiers using both human-labeled and GPT-only annotations to forecast empathy applicability. The results demonstrate robust performance, surpassing heuristic methods and zero-shot LLM baselines.

However, our error analysis identifies ongoing difficulties, including the detection of implicit distress, ambiguity regarding clinical severity, and contextual hardships. These findings emphasize the necessity for multi-annotator approaches, clinician-guided calibration, and culturally varied annotation practices. Ultimately, EAF offers a structured method for anticipating empathy needs prior to response creation, sets a standard for proactive empathy modeling, and facilitates empathetic dialogue in asynchronous healthcare environments.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC