Curiosity-Driven Exploration via Latent Bayesian Surprise
release_omx4pv5g7fgthhwss7wgfezeka
by
Pietro Mazzaglia, Ozan Catal, Tim Verbelen, Bart Dhoedt
2022
Abstract
The human intrinsic desire to pursue knowledge, also known as curiosity, is
considered essential in the process of skill acquisition. With the aid of
artificial curiosity, we could equip current techniques for control, such as
Reinforcement Learning, with more natural exploration capabilities. A promising
approach in this respect has consisted of using Bayesian surprise on model
parameters, i.e. a metric for the difference between prior and posterior
beliefs, to favour exploration. In this contribution, we propose to apply
Bayesian surprise in a latent space representing the agent's current
understanding of the dynamics of the system, drastically reducing the
computational costs. We extensively evaluate our method by measuring the
agent's performance in terms of environment exploration, for continuous tasks,
and looking at the game scores achieved, for video games. Our model is
computationally cheap and compares positively with current state-of-the-art
methods on several problems. We also investigate the effects caused by
stochasticity in the environment, which is often a failure case for
curiosity-driven agents. In this regime, the results suggest that our approach
is resilient to stochastic transitions.
In text/plain
format
Archived Content
There are no accessible files associated with this release. You could check other releases for this work for an accessible version.
Know of a fulltext copy of on the public web? Submit a URL and we will archive it
2104.07495v2
access all versions, variants, and formats of this works (eg, pre-prints)