Learning to Search with MCTSnets
release_wam6niie35a3blqz6sb2jze5ju
by
Arthur Guez, Théophane Weber, Ioannis Antonoglou, Karen Simonyan,
Oriol Vinyals, Daan Wierstra, Rémi Munos, David Silver
2018
Abstract
Planning problems are among the most important and well-studied problems in
artificial intelligence. They are most typically solved by tree search
algorithms that simulate ahead into the future, evaluate future states, and
back-up those evaluations to the root of a search tree. Among these algorithms,
Monte-Carlo tree search (MCTS) is one of the most general, powerful and widely
used. A typical implementation of MCTS uses cleverly designed rules, optimized
to the particular characteristics of the domain. These rules control where the
simulation traverses, what to evaluate in the states that are reached, and how
to back-up those evaluations. In this paper we instead learn where, what and
how to search. Our architecture, which we call an MCTSnet, incorporates
simulation-based search inside a neural network, by expanding, evaluating and
backing-up a vector embedding. The parameters of the network are trained
end-to-end using gradient-based optimisation. When applied to small searches in
the well known planning problem Sokoban, the learned search algorithm
significantly outperformed MCTS baselines.
In text/plain
format
Archived Files and Locations
application/pdf 2.7 MB
file_x6faj3xxdvhr5kbmhhdm7gso4u
|
arxiv.org (repository) web.archive.org (webarchive) |
1802.04697v1
access all versions, variants, and formats of this works (eg, pre-prints)