An Introductory Survey on Attention Mechanisms in NLP Problems release_ayvyqvklxbgrvc7snqrh53buom

by Dichao Hu

Released as a article .

2018  

Abstract

First derived from human intuition, later adapted to machine translation for automatic token alignment, attention mechanism, a simple method that can be used for encoding sequence data based on the importance score each element is assigned, has been widely applied to and attained significant improvement in various tasks in natural language processing, including sentiment classification, text summarization, question answering, dependency parsing, etc. In this paper, we survey through recent works and conduct an introductory summary of the attention mechanism in different NLP problems, aiming to provide our readers with basic knowledge on this widely used method, discuss its different variants for different tasks, explore its association with other techniques in machine learning, and examine methods for evaluating its performance.
In text/plain format

Archived Files and Locations

application/pdf  735.8 kB
file_qngmhzholbekzhtrkuxdcjl4wu
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2018-11-12
Version   v1
Language   en ?
arXiv  1811.05544v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 7360ceb9-075b-4c6f-867b-239edf19772a
API URL: JSON