Word2Bits - Quantized Word Vectors
release_2wej6h4ygvezlhk2ti5nquqrum
by
Maximilian Lam
2018
Abstract
Word vectors require significant amounts of memory and storage, posing issues
to resource limited devices like mobile phones and GPUs. We show that high
quality quantized word vectors using 1-2 bits per parameter can be learned by
introducing a quantization function into Word2Vec. We furthermore show that
training with the quantization function acts as a regularizer. We train word
vectors on English Wikipedia (2017) and evaluate them on standard word
similarity and analogy tasks and on question answering (SQuAD). Our quantized
word vectors not only take 8-16x less space than full precision (32 bit) word
vectors but also outperform them on word similarity tasks and question
answering.
In text/plain
format
Archived Files and Locations
application/pdf 528.3 kB
file_5mc7uijdrbhqnjnex3yo52nvmq
|
arxiv.org (repository) web.archive.org (webarchive) |
1803.05651v2
access all versions, variants, and formats of this works (eg, pre-prints)