Title: Leveraging Video Prediction Models for Cross-Domain Reinforcement Learning from Videos

Abstract: Acquiring reinforcement learning policies from expert demonstrations is particularly difficult when the source and target domains are visually dissimilar, primarily because of missing reward signals and significant domain shifts. To address this, we present XIPER (Cross-domain Video Prediction Reward), a novel reward modeling framework designed to facilitate learning from expert videos captured in domains with distinct visual characteristics. This approach is especially relevant when the agent’s appearance diverges from the expert’s due to variations in color, morphology, or the sim-to-real gap. Specifically, XIPER employs a cross-domain video prediction model to transform agent observations into the expert’s visual domain, utilizing the likelihood of the resulting predictions as a reward signal. Our evaluation on the DMC Color Suite (comprising 8 tasks) and the DMC Body Suite (3 tasks) demonstrates that XIPER consistently surpasses baseline methods, even in the presence of substantial domain discrepancies. Furthermore, analysis on a sim-to-real transfer dataset confirms that XIPER generates effective reward signals for real-world robot observations, relying solely on simulated expert videos. The project webpage provides access to the code, pretrained models, datasets, and video demonstrations: https://sites.google.com/view/xiper

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC