Induction of Subgoal Automata for Reinforcement Learning release_h7vthmu54vdr5jkswajeoeckpu

by Daniel Furelos-Blanco, Mark Law, Alessandra Russo, Krysia Broda and Anders Jonsson

Released as a article .

2019  

Abstract

In this work we present ISA, a novel approach for learning and exploiting subgoals in reinforcement learning (RL). Our method relies on inducing an automaton whose transitions are subgoals expressed as propositional formulas over a set of observable events. A state-of-the-art inductive logic programming system is used to learn the automaton from observation traces perceived by the RL agent. The reinforcement learning and automaton learning processes are interleaved: a new refined automaton is learned whenever the RL agent generates a trace not recognized by the current automaton. We evaluate ISA in several gridworld problems and show that it performs similarly to a method for which automata are given in advance. We also show that the learned automata can be exploited to speed up convergence through reward shaping and transfer learning across multiple tasks. Finally, we analyze the running time and the number of traces that ISA needs to learn an automata, and the impact that the number of observable events has on the learner's performance.
In text/plain format

Archived Files and Locations

application/pdf  1.5 MB
file_kpxvtodubzfczn4mu7nrzmjgnm
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2019-11-29
Version   v1
Language   en ?
arXiv  1911.13152v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: b86010d0-6aeb-4f5c-b092-0a2d0bbefae4
API URL: JSON