Title: MotionDreamer: Enabling Universal Skeletal Motion Generation for 3D Rigged Models

Abstract

Generating motion for rigged shapes is a critical component in the scalable production of 4D assets. However, current template-based approaches are constrained by rigid topologies and struggle to generalize across varied morphologies. On the other hand, per-case optimization methods are not only computationally intensive and prone to local optima but also highly vulnerable to ambiguities arising from different viewpoints.

To address these challenges, this paper introduces MotionDreamer, a diffusion-based framework capable of category-agnostic skeletal animation generation driven by 2D video guidance. To mitigate the shortage of high-quality training data, we have assembled a large-scale dynamic dataset containing roughly 20,000 diverse 3D models. Each model in this collection is equipped with full textures, skeletal rigging, and extensive animation sequences.

We propose a structural-semantic injection mechanism to bridge the kinematic disconnect between 2D visual motion cues and the heterogeneous nature of 3D skeletal structures. By integrating texture and semantic attributes directly into skeletal joint representations, the model can effectively map perceived visual dynamics to specific joint hierarchies and their corresponding functional roles. This capability allows MotionDreamer to produce high-fidelity animations that preserve anatomical consistency across a broad spectrum of unseen categories, ranging from real-world biological species to imaginary creatures.

Comprehensive experiments show that our method significantly surpasses existing techniques, establishing a new state-of-the-art benchmark for robust and efficient 4D asset generation. The source code will be released publicly upon acceptance.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC