File(s) under permanent embargo

Reason: The file associated with this record is under embargo until publication, in accordance with the publisher's self-archiving policy. The full text may be available through the publisher links provided above.

Building High-fidelity Human Body Models from User-generated Data

journal contribution
posted on 29.05.2020, 09:48 by Z Xu, W Chang, Y Zhu, L Dong, Huiyu Zhou, Q Zhang

We propose a key point-based approach, refers to as KPhub-PC, to estimate high-fidelity human body models from low quality point clouds acquired with an affordable 3D scanner and a variation KPhub-I that can achieve the same purpose based on low-resolution single images taken by smartphones. In KPhub-PC, a sparse set of key points is annotated to guide the deformation of a parametric 3D human body model SMPL and then a high-fidelity human body model that can explain the target point cloud is built. Besides building 3D human body models from point clouds, KPhub-I is designed to estimate accurate 3D human body models from single 2D images. The SMPL model is fitted to 2D joints and the boundary of the human body which are detected using CNN based methods automatically. Considering that people are in stable poses at most of the time, a stable pose prior is defined from CMU motion capture dataset for further improving accuracy. Intensive experiments demonstrate that in both types of user-generated data, the proposed approaches can build believable and animatable human body models robustly. Our approach outperforms the state-of-the-arts in the accuracy of both human body shape and pose estimation.

Funding

This work is supported in part by the Science and Technology Research Program of Chongqing Municipal Education Commission under Grant NO.KJQN201900628, and in part by Natural Science Foundation of Chongqing under No. E021D2019034. We would like to thank the Innovation Team on Digital Fashion and Spatial Visual Communication in School of Art & Design, Beijing Institute of Fashion Technology, for providing the original images of the IM-hub dataset of various participants and collecting the body physical measurements. (Corresponding author: Qianni Zhang.)

History

Citation

IEEE Transactions on Multimedia (2020) In Press

Version

AM (Accepted Manuscript)

Published in

IEEE Transactions on Multimedia

Publisher

Institute of Electrical and Electronics Engineers

issn

1520-9210

Acceptance date

29/05/2020

Copyright date

2020

Language

en

Publisher version

TBA