Sample Complexity Bounds for Recurrent Neural Networks with Application to Combinatorial Graph Problems release_4dcgzxbgffcddmca6oaxwdvpxm

by Nil-Jana Akpinar, Bernhard Kratzwald, Stefan Feuerriegel

Released as a article .

2019  

Abstract

Learning to predict solutions to real-valued combinatorial graph problems promises efficient approximations. As demonstrated based on the NP-hard edge clique cover number, recurrent neural networks (RNNs) are particularly suited for this task and can even outperform state-of-the-art heuristics. However, the theoretical framework for estimating real-valued RNNs is understood only poorly. As our primary contribution, this is the first work that upper bounds the sample complexity for learning real-valued RNNs. While such derivations have been made earlier for feed-forward and convolutional neural networks, our work presents the first such attempt for recurrent neural networks. Given a single-layer RNN with a rectified linear units and input of length b, we show that a population prediction error of ε can be realized with at most Õ(a^4b/ε^2) samples. We further derive comparable results for multi-layer RNNs. Accordingly, a size-adaptive RNN fed with graphs of at most n vertices can be learned in Õ(n^6/ε^2), i.e., with only a polynomial number of samples. For combinatorial graph problems, this provides a theoretical foundation that renders RNNs competitive.
In text/plain format

Archived Files and Locations

application/pdf  462.6 kB
file_rnxl7xfhqfe5xhd5az2f3wbzse
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2019-11-19
Version   v2
Language   en ?
arXiv  1901.10289v2
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 70999767-8e6b-4964-b478-ac2e3697b4a6
API URL: JSON