Augment and Reduce: Stochastic Inference for Large Categorical Distributions release_rwhyfrmacvacjnqstg23bdhcgy

by Francisco J. R. Ruiz, Michalis K. Titsias, Adji B. Dieng, David M. Blei

Released as a article .

2018  

Abstract

Categorical distributions are ubiquitous in machine learning, e.g., in classification, language models, and recommendation systems. However, when the number of possible outcomes is very large, using categorical distributions becomes computationally expensive, as the complexity scales linearly with the number of outcomes. To address this problem, we propose augment and reduce (A&R), a method to alleviate the computational complexity. A&R uses two ideas: latent variable augmentation and stochastic variational inference. It maximizes a lower bound on the marginal likelihood of the data. Unlike existing methods which are specific to softmax, A&R is more general and is amenable to other categorical models, such as multinomial probit. On several large-scale classification problems, we show that A&R provides a tighter bound on the marginal likelihood and has better predictive performance than existing approaches.
In text/plain format

Archived Files and Locations

application/pdf  692.5 kB
file_5hira52jvvhwxjostjxkiwg2dq
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2018-02-12
Version   v1
Language   en ?
arXiv  1802.04220v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: bb8271e2-722c-4bdb-9432-839ec6e322b0
API URL: JSON