Title: Investigating Language Bias Amidst Contradictory Data in Multilingual Large Language Models

Abstract:

Previous research has demonstrated that Large Language Models (LLMs) exhibit biases when synthesizing conflicting information to generate answers. This study investigates whether such biases extend to the specific languages associated with each contradictory data point. To explore this, we adapted the "conflicting needles in a haystack" framework for multilingual environments, conducting extensive evaluations using realistic news data across five languages and various multilingual LLMs of differing scales.

Our results indicate that nearly all tested models, including GPT-5.2, predominantly disregard the contradiction, confidently presenting only one potential answer in the vast majority of instances. We observed a consistent preference for certain languages across different models and prompt languages. Specifically, there is a general bias against Russian, while Chinese is favored, particularly in contexts involving the longest sequence lengths. These language preferences remain consistent between models trained within mainland China and those trained elsewhere, although the effect is slightly more pronounced in the former group. Additionally, models tend to prioritize information that aligns with the language of the prompt. We aim to raise awareness among users and developers of multilingual LLMs regarding this specific type of bias, encouraging further research into its underlying causes and potential mitigation strategies.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC