Memory-Augmented Reinforcement Learning for Image-Goal Navigation
release_v5mva4caxndu7k77iro5slgnhu
by
Lina Mezghani, Sainbayar Sukhbaatar, Thibaut Lavril, Oleksandr Maksymets, Dhruv Batra, Piotr Bojanowski, Karteek Alahari
2022
Abstract
In this work, we present a memory-augmented approach for image-goal
navigation. Earlier attempts, including RL-based and SLAM-based approaches have
either shown poor generalization performance, or are heavily-reliant on
pose/depth sensors. Our method is based on an attention-based end-to-end model
that leverages an episodic memory to learn to navigate. First, we train a
state-embedding network in a self-supervised fashion, and then use it to embed
previously-visited states into the agent's memory. Our navigation policy takes
advantage of this information through an attention mechanism. We validate our
approach with extensive evaluations, and show that our model establishes a new
state of the art on the challenging Gibson dataset. Furthermore, we achieve
this impressive performance from RGB input alone, without access to additional
information such as position or depth, in stark contrast to related work.
In text/plain
format
Archived Content
There are no accessible files associated with this release. You could check other releases for this work for an accessible version.
Know of a fulltext copy of on the public web? Submit a URL and we will archive it
2101.05181v4
access all versions, variants, and formats of this works (eg, pre-prints)