Recurrent Reinforcement Learning: A Hybrid Approach
release_ednnw4oa3vds7hurlcgmms5mr4
by
Xiujun Li, Lihong Li, Jianfeng Gao, Xiaodong He, Jianshu Chen, Li
Deng, Ji He
2015
Abstract
Successful applications of reinforcement learning in real-world problems
often require dealing with partially observable states. It is in general very
challenging to construct and infer hidden states as they often depend on the
agent's entire interaction history and may require substantial domain
knowledge. In this work, we investigate a deep-learning approach to learning
the representation of states in partially observable tasks, with minimal prior
knowledge of the domain. In particular, we propose a new family of hybrid
models that combines the strength of both supervised learning (SL) and
reinforcement learning (RL), trained in a joint fashion: The SL component can
be a recurrent neural networks (RNN) or its long short-term memory (LSTM)
version, which is equipped with the desired property of being able to capture
long-term dependency on history, thus providing an effective way of learning
the representation of hidden states. The RL component is a deep Q-network (DQN)
that learns to optimize the control for maximizing long-term rewards. Extensive
experiments in a direct mailing campaign problem demonstrate the effectiveness
and advantages of the proposed approach, which performs the best among a set of
previous state-of-the-art methods.
In text/plain
format
Archived Files and Locations
application/pdf 441.0 kB
file_sibhf3iflrdivitwgqmrfnuzwi
|
arxiv.org (repository) web.archive.org (webarchive) |
1509.03044v2
access all versions, variants, and formats of this works (eg, pre-prints)