Title: Assessing Sample-Size Scaling in African Language Natural Language Inference

Abstract: There is significant scarcity of annotated data for African languages, and it remains uncertain whether simply increasing the volume of annotations consistently improves downstream model performance. This paper presents a systematic investigation into sample-size scaling for Natural Language Inference (NLI) across 16 African languages, utilizing the AfriXNLI benchmark. Operating under controlled experimental conditions, we evaluated two multilingual transformer models—XLM-R Large (fine-tuned on XNLI) and AfroXLM-R Large—both containing approximately 0.6 billion parameters. The models were tested using labeled example counts ranging from 50 to 500, with final results averaged over multiple random subsampling runs. Contrary to the conventional assumption that performance increases monotonically with more data, our findings reveal scaling behaviors that are highly sensitive to language and frequently non-monotonic. We observed that certain languages exhibit early performance saturation or even degradation as sample size grows, alongside substantial variance in low-resource settings. These outcomes suggest that data volume alone cannot ensure reliable performance gains for African NLI tasks, highlighting the urgent need for language-specific dataset development and more robust multilingual modeling approaches.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC