Hyperbolic Representation Learning for Fast and Efficient Neural
Question Answering
release_f6nfijpttfeyzje5shllopborm
by
Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2017
Abstract
The dominant neural architectures in question answer retrieval are based on
recurrent or convolutional encoders configured with complex word matching
layers. Given that recent architectural innovations are mostly new word
interaction layers or attention-based matching mechanisms, it seems to be a
well-established fact that these components are mandatory for good performance.
Unfortunately, the memory and computation cost incurred by these complex
mechanisms are undesirable for practical applications. As such, this paper
tackles the question of whether it is possible to achieve competitive
performance with simple neural architectures. We propose a simple but novel
deep learning architecture for fast and efficient question-answer ranking and
retrieval. More specifically, our proposed model, HyperQA, is a
parameter efficient neural network that outperforms other parameter intensive
models such as Attentive Pooling BiLSTMs and Multi-Perspective CNNs on multiple
QA benchmarks. The novelty behind HyperQA is a pairwise ranking
objective that models the relationship between question and answer embeddings
in Hyperbolic space instead of Euclidean space. This empowers our model with a
self-organizing ability and enables automatic discovery of latent hierarchies
while learning embeddings of questions and answers. Our model requires no
feature engineering, no similarity matrix matching, no complicated attention
mechanisms nor over-parameterized layers and yet outperforms and remains
competitive to many models that have these functionalities on multiple
benchmarks.
In text/plain
format
Archived Files and Locations
application/pdf 1.2 MB
file_fsl3grmmkbadjoxkatptqkmh4u
|
arxiv.org (repository) web.archive.org (webarchive) |
1707.07847v3
access all versions, variants, and formats of this works (eg, pre-prints)