Title: Enhancing Human Interpretation in Migration News Framing through Structured Chain-of-Thought with LLMs

Abstract

Analyzing frames within migration news reporting is a task with significant social implications. For media scholars and researchers investigating how migration is narrated, there is a pressing need for tools that offer not only accuracy but also transparency, auditability, and accessibility, particularly given the resource limitations common in academic research settings. Current LLM-based methodologies often depend on proprietary APIs and large-scale models, which introduce concerns regarding data privacy, reproducibility, and equitable access for media researchers. This study explores the utility of a locally deployable, open-source LLM as an assistive instrument for interpretable frame analysis.

We propose a Structured Chain-of-Thought (SCoT) prompting strategy utilizing the Llama3-8B model. This approach facilitates step-by-step justifications anchored in predefined framing categories. By adopting this structured design, users are empowered to audit model outputs and scrutinize alternative interpretations, a crucial feature given the subjective nature of the task. Our evaluation on a dataset of migration-related news demonstrates that SCoT enhances classification performance compared to zero-shot and few-shot baselines, all while remaining computationally feasible on a single GPU.

Furthermore, we performed a human-centered evaluation where annotators assessed the coherence and impact of the "model's reasoning." The results suggest that SCoT explanations are generally viewed as logical, achieving a mean score of 4.1 out of 5, although there was significant variation across different texts. These explanations also appear capable of prompting reflection on initial interpretations, even in cases where disagreement remains.

Our findings underscore both the opportunities and challenges associated with LLM-assisted frame analysis. While structured reasoning can enhance the traceability of model outputs and aid critical interpretation, it may also subtly influence human judgment. By prioritizing local deployment and human-in-the-loop interaction, this work contributes to the ongoing discourse on developing responsible and accessible computational tools for examining socially impactful media narratives.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC