Human Action Transfer Based on 3D Model Reconstruction release_yg2zoebs75buleufhqwm3zrrky

by Shanyan Guan, Shuo Wen, Dexin Yang, Bingbing Ni, Wendong Zhang, Jun Tang, Xiaokang Yang

Published in PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE by Association for the Advancement of Artificial Intelligence (AAAI).

2019   Volume 33, p8352-8359

Abstract

We present a practical and effective method for human action transfer. Given a sequence of source action and limited target information, we aim to transfer motion from source to target. Although recent works based on GAN or VAE achieved impressive results for action transfer in 2D, there still exists a lot of problems which cannot be avoided, such as distorted and discontinuous human body shape, blurry cloth texture and so on. In this paper, we try to solve these problems in a novel 3D viewpoint. On the one hand, we design a skeleton-to-3D-mesh generator to generate the 3D model, which achieves huge improvement on appearance reconstruction. Furthermore, we add a temporal connection to improve the smoothness of the model. On the other hand, instead of directly utilizing the image in RGB space, we transform the target appearance information into UV space for further pose transformation. Specially, unlike conventional graphics render method directly projects visible pixels to UV space, our transformation is according to pixel's semantic information. We perform experiments on Human3.6M and HumanEva-I to evaluate the performance of pose generator. Both qualitative and quantitative results show that our method outperforms methods based on generation method in 2D. Additionally, we compare our render method with graphic methods on Human3.6M and People-snapshot. The comparison results show that our render method is more robust and effective.
In application/xml+jats format

Archived Files and Locations

application/pdf  3.5 MB
file_fkexufsbvfeuvkpxux5krcir5e
aaai.org (web)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2019-07-17
Proceedings Metadata
Not in DOAJ
Not in Keepers Registry
ISSN-L:  2159-5399
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: f99a6330-718e-4e0d-89cd-fb0781176f79
API URL: JSON