Recurrent Memory Array Structures release_c7m2dn7lbjba5j76o73vvtaoli

by Kamil Rocki

Released as a article .

2016  

Abstract

The following report introduces ideas augmenting standard Long Short Term Memory (LSTM) architecture with multiple memory cells per hidden unit in order to improve its generalization capabilities. It considers both deterministic and stochastic variants of memory operation. It is shown that the nondeterministic Array-LSTM approach improves state-of-the-art performance on character level text prediction achieving 1.402 BPC on enwik8 dataset. Furthermore, this report estabilishes baseline neural-based results of 1.12 BPC and 1.19 BPC for enwik9 and enwik10 datasets respectively.
In text/plain format

Archived Files and Locations

application/pdf  1.4 MB
file_nvzqcawn7vd5nbzhqkyrzv6b7y
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2016-10-23
Version   v3
Language   en ?
arXiv  1607.03085v3
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 1759146c-329d-4632-921f-c85a2a3146b5
API URL: JSON