Title: Assessing Social Textual Engagement and Resonance with Community Awareness: A Human-Centric Approach to Evaluating User-Generated Content

Abstract:

Conventional Video Quality Assessment (VQA) methodologies tend to concentrate exclusively on aesthetic fidelity, thereby neglecting the intricate social dynamics that are central to defining quality within User-Generated Content (UGC). This study advocates for a fundamental paradigm shift, moving away from metrics focused solely on signals toward an assessment framework centered on human-centric resonance. We introduce CASTER (Community-Aware Assessment of Social Textual Engagement and Resonance), a novel task designed to determine if a UGC piece successfully generates positive community resonance. This evaluation relies on the item’s multimodal characteristics rather than its visual quality alone.

To tackle this challenge, we developed MEDEA (Multimodal Engagement-Driven Evaluation Architecture), which incorporates a pioneering Social Chain-of-Thought (Social-CoT) mechanism. In contrast to standard logical CoT approaches, Social-CoT facilitates multimodal perspective-taking by generating a variety of viewer personas. These personas simulate the collective cognitive and emotional responses—referred to as the "community mind"—prior to forming a final quality judgment. MEDEA undergoes a two-stage training process that combines supervised fine-tuning with process-supervised reinforcement learning, utilizing a Social Alignment Reward to ensure that reasoning trajectories remain rooted in genuine human social cognition.

To facilitate research in this area, we have released CASTER-Bench, a thorough benchmark annotated by humans that spans a wide array of UGC categories. Our experimental results show that MEDEA substantially surpasses current state-of-the-art baselines on the CASTER-Bench. Furthermore, the model delivers interpretable and empathetic reasoning paths that closely mirror actual community feedback.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC