SynCred-Bench: Evaluating the Credibility of Synthetic Content in AI-Generated Visual Misinformation

Abstract:

The emergence of advanced generative models has enabled the creation of visual artifacts featuring realistic text and layouts, giving rise to a novel threat known as "synthetic credibility." To address this, we present SYNCRED-Bench, a comprehensive benchmark comprising 600 AI-generated misinformation images. This dataset is evenly distributed across six credible-form categories and seven distinct fine-grained circulation styles. Additionally, we introduce FP450, a collection of real images designed to serve as a negative set for assessing false positive rates.

Our extensive evaluations reveal that current detection systems are largely ineffective. When constrained to a 5% false-positive rate, 15 multimodal large language models (MLLMs) achieved a true positive rate (TPR) of just 10.5%. Open-source AIGC detectors performed even worse, with TPRs below 5%, while commercial APIs reached 57.6%. Human annotators also faced significant difficulties, managing only a 63% TPR in identifying synthetic credibility. These results highlight synthetic credibility as a critical, yet under-researched, challenge in visual misinformation. The proposed benchmark aims to facilitate the development of more robust detectors capable of reasoning beyond superficial indicators of credibility.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC