Sample Complexity Bounds for Recurrent Neural Networks with Application
to Combinatorial Graph Problems
release_4dcgzxbgffcddmca6oaxwdvpxm
by
Nil-Jana Akpinar, Bernhard Kratzwald, Stefan Feuerriegel
2019
Abstract
Learning to predict solutions to real-valued combinatorial graph problems
promises efficient approximations. As demonstrated based on the NP-hard edge
clique cover number, recurrent neural networks (RNNs) are particularly suited
for this task and can even outperform state-of-the-art heuristics. However, the
theoretical framework for estimating real-valued RNNs is understood only
poorly. As our primary contribution, this is the first work that upper bounds
the sample complexity for learning real-valued RNNs. While such derivations
have been made earlier for feed-forward and convolutional neural networks, our
work presents the first such attempt for recurrent neural networks. Given a
single-layer RNN with a rectified linear units and input of length b, we
show that a population prediction error of ε can be realized with
at most Õ(a^4b/ε^2) samples. We further derive
comparable results for multi-layer RNNs. Accordingly, a size-adaptive RNN fed
with graphs of at most n vertices can be learned in
Õ(n^6/ε^2), i.e., with only a polynomial number
of samples. For combinatorial graph problems, this provides a theoretical
foundation that renders RNNs competitive.
In text/plain
format
Archived Files and Locations
application/pdf 462.6 kB
file_rnxl7xfhqfe5xhd5az2f3wbzse
|
arxiv.org (repository) web.archive.org (webarchive) |
1901.10289v2
access all versions, variants, and formats of this works (eg, pre-prints)