Word2Bits - Quantized Word Vectors release_2wej6h4ygvezlhk2ti5nquqrum

by Maximilian Lam

Released as a article .

2018  

Abstract

Word vectors require significant amounts of memory and storage, posing issues to resource limited devices like mobile phones and GPUs. We show that high quality quantized word vectors using 1-2 bits per parameter can be learned by introducing a quantization function into Word2Vec. We furthermore show that training with the quantization function acts as a regularizer. We train word vectors on English Wikipedia (2017) and evaluate them on standard word similarity and analogy tasks and on question answering (SQuAD). Our quantized word vectors not only take 8-16x less space than full precision (32 bit) word vectors but also outperform them on word similarity tasks and question answering.
In text/plain format

Archived Files and Locations

application/pdf  528.3 kB
file_5mc7uijdrbhqnjnex3yo52nvmq
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2018-03-28
Version   v2
Language   en ?
arXiv  1803.05651v2
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: c643b23e-5042-4ab5-afcc-505a445901bf
API URL: JSON