Music-to-Dance Generation with Optimal Transport
release_4wirg55ix5f5vjgkrjcjjrxgtu
by
Shuang Wu, Shijian Lu, Li Cheng
2021
Abstract
Dance choreography for a piece of music is a challenging task, having to be
creative in presenting distinctive stylistic dance elements while taking into
account the musical theme and rhythm. It has been tackled by different
approaches such as similarity retrieval, sequence-to-sequence modeling and
generative adversarial networks, but their generated dance sequences are often
short of motion realism, diversity and music consistency. In this paper, we
propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning
to generate 3D dance choreographs from music. We introduce an optimal transport
distance for evaluating the authenticity of the generated dance distribution
and a Gromov-Wasserstein distance to measure the correspondence between the
dance distribution and the input music. This gives a well defined and
non-divergent training objective that mitigates the limitation of standard GAN
training which is frequently plagued with instability and divergent generator
loss issues. Extensive experiments demonstrate that our MDOT-Net can synthesize
realistic and diverse dances which achieve an organic unity with the input
music, reflecting the shared intentionality and matching the rhythmic
articulation.
In text/plain
format
Archived Files and Locations
application/pdf 29.3 MB
file_n5pta5i2zrbg5agjkb352tkljm
|
arxiv.org (repository) web.archive.org (webarchive) |
2112.01806v1
access all versions, variants, and formats of this works (eg, pre-prints)