An Introductory Survey on Attention Mechanisms in NLP Problems
release_ayvyqvklxbgrvc7snqrh53buom
by
Dichao Hu
2018
Abstract
First derived from human intuition, later adapted to machine translation for
automatic token alignment, attention mechanism, a simple method that can be
used for encoding sequence data based on the importance score each element is
assigned, has been widely applied to and attained significant improvement in
various tasks in natural language processing, including sentiment
classification, text summarization, question answering, dependency parsing,
etc. In this paper, we survey through recent works and conduct an introductory
summary of the attention mechanism in different NLP problems, aiming to provide
our readers with basic knowledge on this widely used method, discuss its
different variants for different tasks, explore its association with other
techniques in machine learning, and examine methods for evaluating its
performance.
In text/plain
format
Archived Files and Locations
application/pdf 735.8 kB
file_qngmhzholbekzhtrkuxdcjl4wu
|
arxiv.org (repository) web.archive.org (webarchive) |
1811.05544v1
access all versions, variants, and formats of this works (eg, pre-prints)