Title: Bridging Human Videos and Robotic Control: A Comprehensive Review of Scalable Vision-Language-Action Learning via Human-Centric Data

Recent advancements in generalizable embodied control have largely been propelled by the large-scale pretraining of Vision-Language-Action (VLA) models. Nevertheless, the majority of current methodologies depend on extensive datasets of robot demonstrations. These datasets are expensive to acquire and are often inextricably linked to specific robotic embodiments. In contrast, human videos are plentiful and encapsulate complex interactions, offering a wealth of semantic and physical insights beneficial for real-world manipulation tasks. However, utilizing these videos directly within VLA frameworks is hindered by discrepancies in embodiment and the common lack of annotations aligned with specific tasks.

This survey offers a cohesive perspective on the conversion of human video data into actionable knowledge for VLA systems. We organize existing strategies into four distinct categories, defined by the type of action-related information they extract: (i) latent action representations, which capture changes between frames; (ii) predictive world models, designed to anticipate subsequent frames; (iii) explicit 2D supervision, which isolates cues within the image plane; and (iv) explicit 3D reconstruction, which retrieves geometric or motion data.

In addition to this classification, we identify three critical open challenges within this field: the organization of unstructured video content into episodes suitable for training; the grounding of supervision derived from videos into actions executable by robots, despite variations in viewpoint and embodiment; and the development of evaluation protocols that more accurately forecast performance and transfer efficiency in real-world deployments. These insights aim to guide future research initiatives. A curated collection of relevant papers and resources can be accessed at https://github.com/AaronFengZY/HumanCentricToVLA-Survey.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC