Title: Enhancing Robustness in In-Context Learning: Utilizing Out-of-Distribution Proxies for Retrieving Demonstrations When the Target Domain Is Unavailable

Abstract:

While prior research indicates that Large Language Models (LLMs) maintain strong performance on Out-of-Distribution (OOD) tasks, this proficiency often degrades significantly as distribution shifts intensify. To counteract this, researchers seek to identify and retrieve informative, distributionally aligned demonstrations from accessible source domains to strengthen LLM inference. However, practical applications frequently involve inaccessible target domains, making it difficult to assess the unknown distribution and consequently compromising the quality of the chosen demonstrations. In response to this challenge, we introduce DOPA, a novel framework for demonstration search that employs an OOD proxy to estimate the inaccessible target domain and steer the retrieval mechanism. By leveraging proxy-based evaluation, DOPA integrates a global diversity constraint grounded in Mahalanobis distance to guarantee adequate variety among the selected demonstrations. Empirical evaluations across various LLMs and tasks confirm that DOPA significantly improves robustness in OOD scenarios\footnote{https://github.com/bort64/ood_code}.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC