Title: Food Noise & False Safety: A Systematic Evaluation of How LLMs Fail to Adapt to Eating Disorder Queries with Clinician Feedback

Abstract: Emerging data indicates a growing trend among individuals with eating disorders (EDs) to turn to Large Language Model (LLM) chatbots for guidance, advice, and emotional support. Despite lacking clinical design, these platforms are frequently utilized as a source of assistance due to their perceived neutrality, accessibility, and expertise, a practice that carries significant risk. This study examines interaction dynamics between ED users and LLMs, specifically addressing the dangers posed by models that blindly accommodate unsafe or self-harming requests. In collaboration with clinical ED specialists, we identify specific linguistic markers in user prompts that correlate with higher rates of unsafe model outputs. By systematically manipulating the level of risk within user prompts, we quantify the extent to which LLMs uncritically adapt to problematic and potentially hazardous inputs.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC