Modelling, Visualising and Summarising Documents with a Single
Convolutional Neural Network
release_7qlivzbg75fszmzxgmlqnwmw24
by
Misha Denil and Alban Demiraj and Nal Kalchbrenner and Phil Blunsom
and Nando de Freitas
2014
Abstract
Capturing the compositional process which maps the meaning of words to that
of documents is a central challenge for researchers in Natural Language
Processing and Information Retrieval. We introduce a model that is able to
represent the meaning of documents by embedding them in a low dimensional
vector space, while preserving distinctions of word and sentence order crucial
for capturing nuanced semantics. Our model is based on an extended Dynamic
Convolution Neural Network, which learns convolution filters at both the
sentence and document level, hierarchically learning to capture and compose low
level lexical features into high level semantic concepts. We demonstrate the
effectiveness of this model on a range of document modelling tasks, achieving
strong results with no feature engineering and with a more compact model.
Inspired by recent advances in visualising deep convolution networks for
computer vision, we present a novel visualisation technique for our document
networks which not only provides insight into their learning process, but also
can be interpreted to produce a compelling automatic summarisation system for
texts.
In text/plain
format
Archived Files and Locations
application/pdf 274.1 kB
file_vmbpqyxf5zbk3ofcwxxj5l2guq
|
arxiv.org (repository) web.archive.org (webarchive) |
1406.3830v1
access all versions, variants, and formats of this works (eg, pre-prints)