Title: Stepping Out of the Crowd: Using LLM-Enhanced Community Notes to Combat Health Misinformation

Abstract:

X (previously known as Twitter) employs Community Notes, a decentralized system for managing misinformation that relies on user participation to flag inaccurate content, provide contextual explanations, and assess the utility of those explanations. Despite its widespread use, our empirical study of 30,800 health-related notes identifies a significant bottleneck: substantial latency. Specifically, the median time required for a note to be designated as "helpful" is 17.6 hours. To enhance the system's agility during spikes in real-world misinformation, we introduce CrowdNotes+, a comprehensive framework powered by Large Language Models (LLMs) designed to accelerate and strengthen health misinformation governance.

CrowdNotes+ operates through two primary mechanisms: (1) the augmentation of notes with grounded evidence, and (2) the automation of note creation guided by utility metrics. These processes are overseen by a hierarchical, three-tiered evaluation system that assesses relevance, factual correctness, and overall helpfulness. To implement and test this framework, we developed HealthNotes, a dataset comprising 1,200 health notes annotated for helpfulness, alongside a specialized model fine-tuned to serve as a helpfulness judge.

Our investigation reveals a critical flaw in current crowd-sourced moderation: voters often mistake stylistic fluency for factual accuracy. By implementing our hierarchical evaluation structure, we demonstrate through experiments involving 15 representative LLMs that CrowdNotes+ significantly surpasses human contributors in terms of note correctness, helpfulness, and the utility of provided evidence.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC