Title: Towards Trustworthy Portrait Editing: Evaluating Demographic Misrepresentation in I2I Models

Abstract:

As instruction-guided image-to-image (I2I) editors become staples in both professional and consumer visual pipelines, their reliability hinges on more than just adhering to prompts; it requires the equitable safeguarding of attributes tied to identity. This study delineates two distinct failure modes: "Soft Erasure," characterized by the faint realization or quiet suppression of intended edits, and "Stereotype Replacement," wherein unrequested, stereotype-aligned demographic traits are introduced. Through a controlled benchmark comprising 5,040 edited portraits, we assess these vulnerabilities across three recent open-weight editors, employing both vision-language model metrics and human evaluation.

The findings reveal that failures in preserving identity are widespread and disproportionately affect different demographic groups. Notably, 62% to 71% of generated outputs display skin lightening. This bias is starkly uneven: Indian and Black source portraits are impacted at rates of 72–75%, whereas White source portraits are affected at only 44%. This disparity points to an output-level drift toward lighter or more White-presenting appearances when identity constraints are not strictly defined.

In a case study focused on mitigation, we demonstrate that adding prompt-level constraints regarding appearance can lower race-change scores for non-White source portraits by as much as 1.48 points, with minimal impact on White source portraits, all without altering the underlying model weights. These results indicate that identity preservation is not a consistent feature of I2I portrait editing systems but rather an unevenly distributed trustworthiness failure with tangible social implications. At scale, these silent distortions have the potential to influence AI-mediated self-representation and exacerbate existing representational inequities. To address this, we propose a controlled audit protocol designed to facilitate fairness-aware evaluation and governance for generative editing systems.

Project page: https://seochan99.github.io/i2i-demographic-bias

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC