babylonpolice.com

Creating pose-driven human avatars is about modeling the mapping from the
low-frequency driving pose to high-frequency dynamic human appearances, so an
effective pose encoding method that can encode high-fidelity human details is
essential to human avatar modeling. To this end, we present PoseVocab, a novel
pose encoding method that encourages the network to discover the optimal pose
embeddings for learning the dynamic human appearance. Given multi-view RGB
videos of a character, PoseVocab constructs key poses and latent embeddings
based on the training poses. To achieve pose generalization and temporal
consistency, we sample key rotations in $so(3)$ of each joint rather than the
global pose vectors, and assign a pose embedding to each sampled key rotation.
These joint-structured pose embeddings not only encode the dynamic appearances
under different key poses, but also factorize the global pose embedding into
joint-structured ones to better learn the appearance variation related to the
motion of each joint. To improve the representation ability of the pose
embedding while maintaining memory efficiency, we introduce feature lines, a
compact yet effective 3D representation, to model more fine-grained details of
human appearances. Furthermore, given a query pose and a spatial position, a
hierarchical query strategy is introduced to interpolate pose embeddings and
acquire the conditional pose feature for dynamic human synthesis. Overall,
PoseVocab effectively encodes the dynamic details of human appearance and
enables realistic and generalized animation under novel poses. Experiments show
that our method outperforms other state-of-the-art baselines both qualitatively
and quantitatively in terms of synthesis quality. Code is available at
https://github.com/lizhe00/PoseVocab.