Towards Learning to Play Piano with Dexterous Hands and Touch
release_vhvll44ltjeupmuo5spyoosp7e
by
Huazhe Xu, Yuping Luo, Shaoxiong Wang, Trevor Darrell, Roberto Calandra
2021
Abstract
The virtuoso plays the piano with passion, poetry and extraordinary technical
ability. As Liszt said (a virtuoso)must call up scent and blossom, and breathe
the breath of life. The strongest robots that can play a piano are based on a
combination of specialized robot hands/piano and hardcoded planning algorithms.
In contrast to that, in this paper, we demonstrate how an agent can learn
directly from machine-readable music score to play the piano with dexterous
hands on a simulated piano using reinforcement learning (RL) from scratch. We
demonstrate the RL agents can not only find the correct key position but also
deal with various rhythmic, volume and fingering, requirements. We achieve this
by using a touch-augmented reward and a novel curriculum of tasks. We conclude
by carefully studying the important aspects to enable such learning algorithms
and that can potentially shed light on future research in this direction.
In text/plain
format
Archived Files and Locations
application/pdf 3.9 MB
file_jdjthqwcprgi3cjtwp2s4lyjsu
|
arxiv.org (repository) web.archive.org (webarchive) |
2106.02040v1
access all versions, variants, and formats of this works (eg, pre-prints)