RRL: Resnet as representation for Reinforcement Learning
release_7j7g4tcvnvhhvdjh5vywdtgtdm
by
Rutav Shah, Vikash Kumar
2021
Abstract
The ability to autonomously learn behaviors via direct interactions in
uninstrumented environments can lead to generalist robots capable of enhancing
productivity or providing care in unstructured settings like homes. Such
uninstrumented settings warrant operations only using the robot's
proprioceptive sensor such as onboard cameras, joint encoders, etc which can be
challenging for policy learning owing to the high dimensionality and partial
observability issues. We propose RRL: Resnet as representation for
Reinforcement Learning -- a straightforward yet effective approach that can
learn complex behaviors directly from proprioceptive inputs. RRL fuses features
extracted from pre-trained Resnet into the standard reinforcement learning
pipeline and delivers results comparable to learning directly from the state.
In a simulated dexterous manipulation benchmark, where the state of the art
methods fail to make significant progress, RRL delivers contact rich behaviors.
The appeal of RRL lies in its simplicity in bringing together progress from the
fields of Representation Learning, Imitation Learning, and Reinforcement
Learning. Its effectiveness in learning behaviors directly from visual inputs
with performance and sample efficiency matching learning directly from the
state, even in complex high dimensional domains, is far from obvious.
In text/plain
format
Archived Files and Locations
application/pdf 8.9 MB
file_b3ubzvdjtzasznqdpnxymr46mu
|
arxiv.org (repository) web.archive.org (webarchive) |
2107.03380v2
access all versions, variants, and formats of this works (eg, pre-prints)